Resources
Dec 19, 2023 |
57 min
|
639 views

Stephanie Lussier @ Moderna | Data Science Hangout

We were recently joined by Stephanie Lussier, Manager, Biostatistics at Moderna to discuss growing a new data science team at Moderna and developing a pipeline to enable internal clinical trial decision-making at rapid speeds.

Speaker bio:
Stephanie Lussier is a Manager of Biostatistics at Moderna. She’s a member of the Moderna Specialty Data Analytics team, which focuses on building a seamless analytic platform and capabilities to enable quantitative decision-making in a timely fashion across clinical programs. Her professional interests include designing novel data visualizations, building tools that help clinical trial statisticians, and promoting the use of open-source software.

______


► Subscribe to Our Channel Here: https://bit.ly/2TzgcOu

Follow Us Here:
Website: https://www.posit.co
LinkedIn: https://www.linkedin.com/company/posit-software/

To join future data science hangouts, add to your calendar here: pos.it/dsh (All are welcome! We'd love to see you!)

Thanks for hanging out with us! 💛

image: thumbnail.jpg

Transcript#

This transcript was generated automatically and may contain errors.

Awesome. Then let's jump in here. Well, welcome to the Data Science Hangout, everybody. Nice to see you all back here. And if this is your first time joining us, a special welcome to you as well. It's so nice to meet you. I'm Rachel. I lead Customer Marketing at Posit.

I like to start asking this, but is this anybody's first Data Science Hangout? Say hi in the chat because we'd love to welcome you in and say hello to anybody joining for the first time too. But this is our open space to chat about data science leadership, questions you're facing, and getting to hear about what's going on in the world of data across different industries. So we're here every Thursday at the same time, same place. If you're watching this recording on YouTube later, the link to add it to your calendar will be in the details below as well. I said every Thursday, but just a heads up, we will be off next week for the Thanksgiving holiday. But at the Hangouts, we're all dedicated to making this a welcoming environment for everyone. Love hearing from everybody, no matter your years of experience, your titles, the languages you work in, or industry. It is totally okay to just listen in here. Maybe you're walking your dog, you're on your lunch break, or just want to listen in for the first one. But there's also ways you can jump in and ask questions or provide your own perspective. So you can jump in by raising your hand on Zoom, and I'll keep my eye out. You can put questions in the Zoom chat, and feel free to put a little star next to it if you want me to read it, or I can call on you to introduce yourself and add some context. And then we also have a Slido link, which Curtis will share in a second here, where you can ask questions anonymously too.

Just before we get started, I wanted to share if you are hiring, feel free to share any roles in the chat here. That's definitely not spammy to us. The Biogen team actually shared one with me earlier today, so I just wanted to share it in the chat there. One more shout out, if anybody is in Boston, Kevin and I are hosting Tidy Tuesday meetup tonight. Yes, on a Thursday, but that's at Microsoft in Cambridge. So I'm just going to put that in there as well.

But with that, I am so excited to introduce my co-host for today, Stephanie Lussier, Manager of Biostatistics at Moderna. And Stephanie, I'd love to have you introduce yourself and share a little bit about your role, and also something you like to do outside of work.

Sure, so thank you for the brief introduction. As you said, I'm currently a Manager of Biostatistics at Moderna, and I sit in our Specialty Data Analytics team. So a little bit about my background, I graduated with a Master's in Biostats in 2017, and then proceeded to work at a CRO, a contract research organization for five years. And during that time, I had what some might call a little bit more of a traditional statistician role within the clinical trial realm, and was doing open source projects, kind of more on the side, like a 20% time kind of deal. And so in 2022, I kind of got to the point where I really wanted that to be a bigger part of my role. And so that's where I found my current position at Moderna. And so I'm in the Specialty Data Analytics team, and our mission, kind of in simple terms, is to just use clinical data with modern open source technologies to really enable internal decision making.

So that's where I am now, and happy to talk more about it. And outside of work, I kind of collect hobbies like Pokemon cards. And as a result of that, I'm definitely like jack of all trades, master of none, which I'm okay with. And so some of those hobbies include yoga. That's kind of a big one for me. I've been learning to play tennis. I make a lot of sourdough. I'm learning to throw pots on the wheel. And I do like to travel. And when I travel, I do like to scuba dive. So I'm happy to talk about hobbies for an entire hour. I love them. I hope everyone has awesome hobbies. Yeah.

Wait, can you tell me a bit more about what throwing pots on the wheel means?

So like using clay to make pottery. And so a lot of that, at my skill level, means making cups and bowls that aren't very symmetrical, but are supposed to be. It's very relaxing. For me, like I can't draw at all or paint or anything of that type. And so this is kind of an art form that I can do a little bit.

I love it. That's awesome. Oh, I wanted to throw in one little tidbit, if you don't mind. So with all due respect to past data leaders, I thought a little bit about what my goal is, being on the Data Science Hangout today. And for those that don't know, the leaders are great, but the chat is where it's at. And so my goal is just for people to have fun in the chat. So go crazy. It has resources. It has debates. It has fun topics. And if you've never participated in the chat before, I'd encourage you to do so if you want to.

The leaders are great, but the chat is where it's at. And so my goal is just for people to have fun in the chat.

Becoming a manager for the first time

I love it. Thank you so much for that call out to the chat. Yes, the chat is always a party. I see the excitement starting already in there. Well, Stephanie, while we are waiting for some questions to come in from everybody here, I know something that we talked a bit about is becoming a manager for the first time. And I'm curious if there's something that you've learned or you had to learn throughout this process of becoming a manager that you'd be able to share with us.

Sure. So a lot of pieces of this. In my previous role, like I said, more as like a traditional statistician, you work within clinical teams. And so you end up doing some management of like statistical outputs. So that is kind of like managing statisticians and programmers on a project level. And I think that was really good preparation for being an actual true manager. And so now I'm on a team. We have seven full time employees and seven contractors and I'm starting to pick up part of like the true management pieces of that.

But let's see, as far as what I've learned, there's a lot of paperwork parts of it. And so hopefully anyone becoming a manager has some sort of mentor walking them through that part of it. For example, requesting a laptop is a pretty difficult goal to achieve, apparently. Shout out to our awesome digital team who helps with that. So learning like those little technical pieces of it is like a whole new world to just take on slowly and and ask questions. I think that's the most important thing. And then as far as actual management, I think just forming a relationship with people, working closely together, asking them to ask questions, just really like creating a comfortable space has been really important for me.

Statistical languages and the R path

Thank you. I jumped in right away with a question. And I know you said you want me to ask it. But the question was, when you graduate or what is the language that you use in your academic program? Did you need to learn SAS when you joined the CRO? The question and maybe acronyms will expand on them a bit too. Yeah, I'm doing my very best with not over using acronyms. So please catch me if I do. Yeah, so in the clinical trial world, the main statistical language that is used right now is SAS. I don't know what SAS stands for. It's fine. But it's a statistical programming language. So during my grad school, I was lucky enough to learn both SAS and R and even some Python. And I think that really guided where I ended up today. So SAS was taught in certain classes that I had and in other classes, the professors favored R. And I think the reason I really ended up on the R path was that my main thesis project was done in R. And so that really gave me a strong foundation so that when I started my first role, I was definitely expected to use SAS in that more traditional setting. But I was able to connect with the people in my first organization who did use R. And that really kicked off thinking more about developing with R and what R in the clinical trial space looks like.

Visualizations for internal decision making

Thank you. Yeah. Something that you and I had chatted a little bit about before is that you've realized people don't want to look at these standard tables, listings, and figures, which I just learned that's what TLFs means when I see that listed or named in pharma. But I'm curious, in building tools for internal decision making, what has been really helpful for you in communicating?

Yeah. So the fun thing too about the TLF abbreviation is that it goes by many abbreviations. So watch out for that. It can be TLF, TFL, TLG, TGL. And it all means the same thing. It was right on top of it in the chat too. And why is it like that? I think that's just to confuse people. And I think that's a common theme with abbreviations in clinical trials. So to actually answer the question, yeah, a big part of my role is creating visualizations for internal decision making. And so we think a lot about what is the best way to do that.

I think one of the most important questions is what is the question that someone is trying to answer and who is the end user? And so, for example, a lot of times the end user for us is, say, the executive committee. And so people on the executive committee at Moderna, at least, are highly skilled professionals. They're not looking for a green button or a red button. They want to know what were the results. And they want to see it in a visually pleasing, straightforward display. But they also still want to know some key statistical elements. And so that's actually part of the reason why a lot of the people on our team have a statistics background is because we are still displaying confidence intervals and risk ratios in our displays. And so that's an example where we know the audience and we know the questions they're going to ask.

Another example would be we create, say, visualizations that are going to be used by the clinical team that want to look at data in an ongoing way. And so you might think about how well those people know this data and they want to know every nitty gritty about it. And so that's where something like interactive applications really comes into play so that as they form their own questions, they can answer those questions.

That's great. Can you share maybe an example for us or example use case where people were able to answer some of their own questions with an interactive app?

Sure. So I'm working, one of the big projects that I work on both internally and externally right now is on patient profiles. And so the whole goal with patient profiles is that you can look at one particular participant in a study and the primary uses there is really for safety. So you could look at, say, their labs, basic demographics information, their what we call adverse events, so any safety signals that happen through the study. And so we had a participant that had an unexpected response in a biomarker. And so they were able to use the patient profile to look at when that particular biomarker uptick occurred, what else was going on for that particular participant.

Thank you. I think Bill's question maybe expands a bit on this. Let's see. Was that visualization, was that a Shiny application? I think Bill's question was, have you developed Shiny-based apps for your internal users? And have you ended up using the same interactive or static visualizations in drill downs when interacting with the FDA? Okay. That's like three questions. Yeah. We can break it up a bit. Yeah. No, that's okay. So much to say here. So

the particular example I provided is actually all being powered by non-Shiny interactivity.

And there's been a couple of great talks about this topic lately. DeepShaw gave a hilarious talk at PositConf about this. And my mentor, Augustine Calatrone, also had a recorded talk at rFarmer where he kind of touched on this as well. And so kind of with a judicious use of crosstalk, plotly, interactive tables, you can make an interactive HTML document that has a limited but pretty good capabilities. And so that is the current platform for that particular patient profile. So yeah, if you're not familiar with that setup, I do think it's powerful.

In my previous company, we didn't really have a good hosting platform. And so we were able to really utilize that setup to create interactive reports. Like at a high level, the limitation is that the interactivity happens on one level, sort of. And there's no recalculation happening of the data. You're essentially filtering the data. But you can really do a lot with that setup.

Choosing the right technology

So then moving to like Shiny, it's definitely something that our team uses. We have a variety of different visualizations. And so that decision is kind of part of like what we talked about earlier when you get, say, a data visualization request deciding what is the right technology. And so if more and more exploration is the goal, then Shiny, I think, is the way to go. And we definitely utilize that.

And then I think the third question is about sending that to the FDA. And that's not something that we're doing right now. But I think probably a lot of folks are following the R submissions group. And they've done a pilot to look at what a Shiny app to the FDA could look like. And they're going to be doing more utilizing the WebR framework, which is a hot, exciting topic right now, that could apply really well for this concept of Shiny to FDA as well. Many answers.

Right. Great job. And I guess probably fun to just throw it out here as well that some of the team from the FDA is actually going to be joining us at the Hangout on February 1. So putting it out there so people know as well. But I see a lot of love for Crosstalk in the chat. It feels like we should have some sort of event on that at some point.

Crosstalk event. Very awesome.

Awesome. Sita, I see you asked a question in the chat. You want to jump in here?

Yeah. Hi. Thanks. So I'm curious from a more management perspective, how did you kind of navigate this transition from a more day-to-day interactions with mobile statisticians who knew the same language and were talking the same way versus to working at Moderna, where you kind of have to interface with a whole different group of people, say biologists, data scientists, a very diverse kind of group of people, assuming that it was less diverse in the sense of having more biostatisticians in your previous role? And kind of what challenges did you have and kind of how did you deal with that? Okay. Yeah, I love that question.

I think it's an awesome part about the clinical trial world is that you do get to interface with people in so many different roles. And I would say that at my previous position, I did interface with a lot of different types of people already. And so maybe that was good preparation, working with clinicians on the trial. I even used to have phone calls with people at the sites. That was always fun to feel more a part of the trial. So I think I definitely already had cross-functional area training. So then moving to Moderna, I think part of the difference is the group that we're in now is very much like a support team. So we have presence on every study. And our role is a little bit like, we're here to help you. How do we help? Does this help you? And so gaining a pretty good understanding of what would help people is really important so that we can deliver tools that are useful. Did that answer your question a little bit? Thank you so much.

Getting useful feedback on reports

Thank you. Russ, I still had a question in the chat. You want to jump in here next?

Sure, thanks. Similar to you, I wind up sending reports up and stuff. And sometimes the feedback that I'll get is good and sometimes like, eh. What's one thing that the leadership does that's good, that's helpful to you that gets them the report that they actually want?

Yeah, I love that. So what is good feedback and what is less helpful feedback?

I'm thinking. Good feedback is for them to tell us how and why a report was or wasn't useful so that we can then apply it to future reports. And so the why I think is important. And maybe you have to ask the why. Because if they just say this didn't help, that's not useful. So getting a better understanding of what they were looking for, maybe an understanding of a report answers the question that they asked, hopefully. But if that question changed over time and that's why the report is no longer helpful, a little bit of insight into how that question might have changed or why that question might have changed helps us kind of predict in the future how questions might change or how we can provide supplemental materials that might answer those changing questions. So yeah, I think the takeaway on that is it's OK to turn it into a conversation and probe a little bit more so that you can move forward and improve for the future.

So yeah, I think the takeaway on that is it's OK to turn it into a conversation and probe a little bit more so that you can move forward and improve for the future.

OK, thank you. Did you have any thoughts you wanted to address?

I'll just make it really quick. I have a lot of prior careers, one of which I'm actually a professional counselor with a master's in the whole thing. So to me, listening is really, really, really critical. I think it's almost more important than the analysis. So just make sure that I really, truly understand what the question is. And if I can give a professional counselor tip for just a second, also find out what it is not. Because when you find out what it is, you might go four miles down that path and find, oh, it's actually something slightly different. I was working on that. No, we don't want that. Is it this and not that? And once you do that, a lot of things clear up, and then it's easy to move forward.

Awesome. Yeah, I previously did a lot of work on manuscript writing. And so in traditional clinical trials, you have your endpoints that are pre-written, and you answer those endpoints that are pre-specified. And sometimes in a more exploratory biomarker manuscript setting, there can be a lot of back and forth like you're talking about with questions changing over time. And so keeping the conversation flowing was always important to really narrow the scope and try to find the answers that they were looking for. I see a lot of great thoughts on this topic, too, in the chat. I don't know if, Catherine, you wanted to share your thoughts here, too.

Yeah, I'll kind of repeat what I said in the chat is one of the things that I learned to ask for is what questions are my leadership being asked that led them to ask me to do something? Because how they're interpreting the questions that they're being asked, and then there's like three levels of information distillation before they ask you for a piece of math or a piece of reporting or a visualization. And so then you make the thing that you think they were asking for, but it doesn't really meet the ask that their leadership asked them for. And so there's a little bit of a cycle there. And so getting plugged into meetings where you can listen to how your leadership talks about these kinds of things or being forwarded emails or things like that can really help you get a broader picture of where your organization fits within the broader thing. So, okay, this is how the reports that I deliver are used by these other teams to get more of a sense of like, okay, what information is actually important to the ultimate decision making?

Yeah, one of the comments that you touched on, Catherine, is something that we kind of prioritize in our team. And that's asking to get invited to the meetings, to kind of parse down the amount of distillation that occurs. And so no one wants a calendar full of meetings when there's lots of, you know, deep work to do. But when they're important, asking to get invited so you can hear the whole conversation instead of the summary.

Lisa, I see you had some nice feedback on this as well. Do you want to do you want to share that?

Sorry, I feel like I wrote a lot today. Are you talking about just like the comparison to feedback from students or? Yeah, yeah, the teaching. Yeah.

Yeah, no, I was just more I don't think I have much to add besides that it just sort of triggered sort of triggered in my mind that it reminded me of teaching. And when I get these responses, especially when I first started teaching that they would be like, you're awful, or you're really nice. And I was like, Oh, I feel really lovely that you think I'm nice, or I feel terrible that you think I'm awful, but neither of those really helped me. So I always ask them to be like as specific as possible. But I've never thought of asking someone at work for that specific feedback. So that was a nice connection is all I was saying.

Thank you for sharing. Yeah, I'm trying, trying to call out some of the chat comments as well. Because I know when people watch the recording, they don't get the advantage of getting to see all these amazing comments, too. I know, let's see, Adrian, you asked the question in the chat a bit earlier. Do you want to jump in?

Recyclability and internal packages

Yeah, let me see if I can recall it and restate it pretty clearly. I'm interested in, Stephanie, at Moderna, and with sort of the profile of work you're doing, how recyclable is sort of those analyses of those reports. And if you want a bit of context, I spend a lot of time trying to not do anything multiple times. So I'm curious if the biostats area, or at least, you know, with your team, if there is a lot of opportunity to library your sort of starting points or what have you?

Yes, great question. The short, very short answer is yes, there's a lot of recyclability. And I think a lot of similar teams, as well as mine, are working on, you know, creating internal packages that help with that recyclability. But something that has kind of been on my mind lately is making sure that when you're creating functions, modules, whatever, you're still allowing for customization. Because at least in my experience, what people in clinical teams, they want to add stuff. They want to say, can we look at this too? Can you add this facet? You know, even little details. And so as we go on, you know, making progress in creating like an internal package system, that's something that we're trying to keep in mind is finding that sweet spot between out of the box, boom, here you go, I made you this plot, and allowing for customization. Thumbs up, awesome.

Advice for recent graduates

I want to jump to a few of the anonymous questions too. And one was, do you have any advice for a recent graduate trying to get her foot in the door? It's a bit hard to do currently. Okay, we're all with you. It's not easy. My job search to switch jobs took what felt like a long time. I don't want to like put a number on it. Like I was always told statisticians can get a new job so quickly. And I was really picky in my situation. And so it wasn't quick. And I did have like, fear surrounding that, that I look back on now. And it all makes

sense, you know? So but for thinking about someone with less experience, I saw the chat going this direction last week, which is a topic that I think is really cool to talk about is the presence of like an online GitHub repository. Because I think there's arguments for and against this. But I think in a position of being a recent grad, that's something that I would think about doing is do some little tiny projects and put them on GitHub just to demonstrate your programming skills.

And then in the clinical trial space, if you're having difficulty, I would suggest looking at contract research organizations, I learned so much in that role. And I think they have a lot of positions, hopefully for more entry level. Like, I hope I'm not making too big of generalizations. I think sometimes the pharma side of thing does look for more experience. And there are more entry level associate type roles in the CRO world. So that's something to like, look into and read more about. And then another thing I would think about is, I don't know exactly your background. But the CR or the clinical trials world has a lot of clinical trials knowledge. And so trying to find some resources online to become familiar with what a clinical trial looks like. We have data standards called CDISC to do a little bit of research about that. I think that would make you really stand out as a candidate.

Interactivity in dashboards

Thank you. Matt M, I see you had a question a bit earlier. Do you want to jump in and read that one? Sure. So I was curious, you talked about some of the dashboards you've built. And I'm just curious, what's the limiting principle for how much interactivity to give people? Because I kind of found that there are two sides to that continuum. One, you can just keep building the app and building the app and adding more functionality. But you want to deliver it. And then the more content you add, the more functionality you add to it, the more possibility someone else pointed out that there's a possibility that people are going to derive insights that aren't necessarily true or meaningful. But then on the flip side, you know that the one thing you don't implement is the thing you're going to get asked for. So where do you draw the line on what to do and what to leave out?

That's a big question. It's difficult to nail down. I think in my particular realm of things, I definitely think about it along the lines of where a study is progressing. So in the early, early phases of a clinical trial, or even preclinical, things can be much more exploratory. And so when exploratory is the goal, hypothesis generating is the goal, you know, go crazy and put a disclaimer that, you know, there are things discoverable here that need to be proven outside of this application. And then when you work your way down to, you know, questions that are more formulated, a researcher has really provided the question they're trying to answer. That's where I would think about limiting it and trying to really just answer the question that's being asked and limit the back and forth. Because, you know, there's a whole theory behind this. But when you're in phase three of a clinical trial, the question should be laid out and the question should be answerable. And that upholds the entire integrity of the trial. So that's kind of how I think about it. No, that's great. Thank you. I also really appreciate the disclaimer. Yeah, use disclaimers.

But when you're in phase three of a clinical trial, the question should be laid out and the question should be answerable. And that upholds the entire integrity of the trial.

I see Abigail and Jesse, you had some thoughts on this topic, too. Do you want to jump in here? I'm not sure if you're in places where you can jump in or not, but I'll give a second here. Oh, was this about interactivity? Yeah. Oh, yeah. No, I was just saying, like, I don't know, I have real dashboards. I don't know. Dashboards are tough because people do, like, you give people self-serve analytics and then they come to conclusions, which are wrong sometimes. So figuring out how to sort of manage that, I think, I don't know, I found that to be difficult. Sometimes I just want to give somebody, like, I wrote you a paper, I made you a PowerPoint thing, as useful as the interactivity definitely can be. Definitely.

Deep work and time management

Kevin, I see you had a question in the chat. Want to jump in here? Sure. Yeah. What was my question? Sorry, I know when there's a lot of chat, it gets hard to remember what our questions were. Yeah, it was about, oh, that's right, now I remember. It was about deep work and as a manager, and the question was basically, like, you know, are you able to manage to carve out time to do deep work still or do you have to delegate most of that, you know, or how do you kind of handle that? Yeah. So generally, I would say that my manager workload, like, workload as a manager is not 40 hours a week in any way right now. And so I definitely find time to carve out for deep work. And I also am not overloaded on meetings, which I know what that looks like from previous experience as well. And so it definitely is doable. I think part of that is because our group is so young. Everyone is heads down, creating stuff, doing work, focusing. And so that kind of contributes to, like, an overall environment where we're all getting in hours for deep work. Yeah.

But do you end up having to hit, you know, attend lots of meetings that they don't have to go to, you know, to interface with your, you know, with the people that you guys are serving, you know, or other management and things like that? Yeah, right now, I really feel like that is being handled well, like the meetings that I'm attending are relevant and helpful. And I could see, of course, how that could change. But, yeah, I think it's something to always keep in the front of your mind is time resourcing. I'm like a fan of time blocking in my calendar. And so always asking the question, do I need to be at this meeting? What am I getting out of this meeting? And you don't always hit the nail on the head, you go to a meeting that you shouldn't have been at, or you miss a meeting that you should have been at. But just always like keeping it in the forefront of your mind, I think is an important skill. Right. Thanks so much.

Yeah. And something I've been thinking a lot about recently as well, like, initially, when you get invited to a meeting with a new team, it feels good, like, oh, they want my point of view in this conversation. And then when I realize, okay, that meeting is every week or every other week, at what point can you then say, okay, no, I don't think I need to be at this meeting every week. And how do you communicate that to people that know you're just not going to attend? Yeah. I think as soon as you're saying to yourself, is this useful? It might not be useful. And so just making it a conversation, I think, but it's not easy. But yeah, I think it's something for any career person to, like, always be thinking about.

I see a few ideas here in the chat. Donald, do you want to jump in? Sure. Just like as far as carving out deep work time, I think I had some questions too, so I'll try and answer those. I usually block out two to three hour chunks multiple times a week. I think it's also recommended in common coursebooks to do that too. And I've been doing it ever since. It's been extremely helpful. I think, Jared, you asked if people ever complained that I'm never available. Sometimes, but I also notify my director who I report to, like, oh, you know, I'm doing this. This is really important because I'm a knowledge worker. I need this time to do that. And he's completely on board with it. So usually we kind of loop in if we need to, a discussion of priorities to see if anything else is to happen.

I see that Frank is on the call, and I don't mean to call on you, but something that's always stuck with me that you said, Frank, is like you color code some of the meetings that you have on your calendar. And, like, the ones that give you energy are a certain color and the ones that are, and maybe I'm putting words in your mouth, but are more draining are a different color. And I thought that was a really interesting approach too. I hope, Frank, your calendar doesn't face all of your coworkers in that case.

But, you know, honestly, even if it does, that's fine with me. And me, like, all right, so most of this came when I was at Target, and Target, huge company, right, like most of us work in huge companies. And, yeah, I mean, in those cases, I was always pretty honest about that stuff. I was like, hey, yeah, I'm going to show up, right? Like, this is a little bit draining for me, but right, I know it's important for me to hear this. I know it's important for me to be here. When you do the color coding, it makes it very easy to see, like, hey, red, red, red, red, red. And then, right, like the last red meeting is something that's really important. And, like, you have to be paying attention and on top of it. And, I mean, all of us love to find patterns in data, right, even if your sample size is, like, really small. And then being able to just, like, reflect on that and notice that, oh, man, there's no way I'm going to be able to give my focus and attention and 100% at, like, this really important meeting to, like, the one-on-one with a team member or, right, like the big meeting with a VP. If you can know how you will be and then sharing that with other people around you, like, people are surprisingly or maybe not surprisingly, like, willing to adjust. So, I've always been a fan of that visibility. If it rubs people the wrong way, it's going to be even worse for the whole organization if, like, you show up to this really important meeting and you're drained. So, it's been my philosophy.

Okay. I'm thinking of, like, a scatterplot where we're plotting, like, importance versus passion, maybe. And so, really important, no passion, kind of still got to go. Less important, tons of passion when there's time for it. Like, we have quadrants. Is that along the lines of your thought process? Yeah, but the way you describe that and the way I'm visualizing it, like, none of this stuff lives in a silo, right? Like, it's all connected and especially, like, the series of meetings leading up to something important before, like, you can go on a walk if you're in Minneapolis and in the Skyway and, like, re-energize yourself. So, I'm not exactly sure. Maybe we can, like, put together a three-dimensional scatterplot so we can see, right? Like, time would be your other axis. Time of day is important and that's your third dimension. Okay, very cool.

Asking why before jumping to solutions

We should, we should write that one up. Yeah. Mike, I see you had a question earlier on, as a consultant, when somebody asks, can you help me do X? Do you want to jump in and continue that question? I think that was very much related to what you were talking about previously on the why is someone asking for a report? But it's, it would be interesting to hear what Stephanie thinks that, you know, sometimes I get these messages that go, Mike, can you get this package for me? Or could you show me how to do that thing? Without any context and, you know, as a consultant, you might think, well, I'll just jump in and I'll help. I'll just do it, you know, get that thing. Get that thing. But then if you unpick it, you realize that actually, you know, there's a whole world of, you know, either there there's a better solution or, you know, there's a simpler thing that they could do or, you know, it's related to, you know, back to the why question.

So, you know, it's, you can be the helpful consultant or you can actually do something that makes that, you know, eliminates tech debt or makes it easier to maintain or something. Yeah, I think I'm coming to mind with examples just within my own team of getting like a ping saying, how do I, Stephanie, how do I make this access log 10? And it's probably a good idea to first ask why are you making that access log 10? These kinds of things to try to make sure you're solving the right question. Yeah, I like that.

Brennan, I see you just asked a question. Want to jump in here next? Oh, yes, ma'am. The question I want to ask is just what's the most fulfilling project you've worked on so far in your career? Oh, I really like that question. It, like, gives me a good moment to reflect back. Like, you know, we talk about data science and

clinical trials, but clinical trials are like a big deal. They affect people's lives. And that definitely is a huge motivator for me in my day to day. And it's always important to like, think back, like this data is a person giving up their time, giving up their selves to help improve science. And like, I'm giving myself chills. Like, it's just such an awesome thing to be a part of.

And so there's so much cool stuff happening at Moderna that could make very big advancements in the science world. But me personally, my answer definitely has to be that in my previous role, I worked on a really big food allergy trial. And it would essentially be the first actual treatment, like a antibody approved to treat food allergy. And it was a behemoth of a study. No one wanted to work on it. It was very complicated. But it was so rewarding to think that you're helping children who can't have lunch in the cafeteria change their lives. And so I was really proud of the work I did on that study. It was an awesome team. And they've completed it without me. So yeah, it was a really good experience.

this data is a person giving up their time, giving up their selves to help improve science. And like, I'm giving myself chills. Like, it's just such an awesome thing to be a part of.

Oh, and I'll add to that. I had a colleague. Once I was off the study, I had a former colleague whose child joined the study. So that's super cool to actually like, feel so connected to that study. And so again, hearing from his point of view, oh my gosh, we're at the doctor's all the time. This is so much work. And like, on our side, that's like, this schedule of events is insane. I don't know how to program the visits data set. Like those two things go hand in hand, and our side is easier. Like I think always keep that in mind.

Yeah, thanks for asking that. Yeah, thank you. I love that question, Brennan.

Natalia, I see you had a question earlier in the Hangout. You want to jump in here?

Yeah, sure. We are on such a great note. I don't know, like, if switching back to programming.

Open source and Pharmaverse at Moderna

Yeah, but thank you. So you mentioned that Moderna is kind of open to open source and and those tools. So I'm very curious if in your day-to-day work, you're using any like Pharmaverse packages or any other pharma-related open source packages? And like overall, how are you approaching the open source projects at Moderna? Are you going to release some open source yours like as well within your company? So stuff like that.

Yeah, great. We love like, I mean, I love open source. I think that's why we're here on this Hangout, because we all love open source. So yeah, your question had a lot of different parts again. So as far as our team using Pharmaverse, so a little background for non-pharma people, Pharmaverse is a collection of packages contributed to by all different pharma companies as well as CROs. And the aim of the Pharmaverse packages is largely to do with submission. So actually putting a drug in front of the FDA for approval. And that is not where my group is using open source software at this time.

We're very much keeping our eyes and ears open to it. And if that's something you're interested in and you haven't yet watched Nova Nordic's YouTube video, someone can put it in the chat. That's like very groundbreaking, very exciting. It was presented beautifully. And we're very happy at Moderna to see these companies laying the path for us. Very appreciative of those companies as well as all the initiatives that surround that. So yeah, that's not totally our focus. And so I don't see us using Pharmaverse packages a ton.

That being said, we have a safety stats group that are also open source folks, and they have developed similar table packages for what looks more like an FDA reporting output. And again, those are currently being used internally. But as a part of that work, and this is to answer another part of your question, there is a member of that team or several members of that team that do participate in Pharmaverse open source, like the Falcon package. And so in general, I find that leadership is very open to open source collaboration.

Statisticians, programmers, and career advice

Thank you. Let's see. If I know as we get to the end of the time here, if you have a question that we maybe missed in the chat among all the amazing conversation in there, please raise your hand here so we can get to it as well. But Jared, I see you had a question a little bit earlier, if you want to jump in next.

Sure. Can you guys hear me okay? I guess like in the CRO space, you bring up a few good points, but I think for me, it's more like transitioning and towing the line between a statistician or programming role. I've seen a lot more individual struggle going from a programmer to a statistician than from a statistician to a programmer. And so the thing that I keep finding myself wondering is do you find yourself missing some programming aspects where you're kind of just inundated with writing up SAPs and protocols versus actually getting to do some of the work? And then from that same kind of same coin, going back and thinking the role that you're in as a statistician, if it's more of a standard role or one of discovery, and trying to stay in that niche or being more kind of fluid in the roles that you take on, like you said, you're very picky about that for that. I didn't know if that had partially had to do with the reason.

Yeah, really great question. It's interesting to see, like I've seen colleagues over time switching between programming and statistics. And that's always kind of telling about a person's passion and how that might change over time. I think it's really cool. I actually think right now in my current role, I have a very good mix of statistics, programming, management, kind of all blended together, which for me works well. I like doing all of those things.

And like to your a couple of things. So I think we're going to see maybe the definition between statisticians and programmers change over time, like where that line is drawn. And that future roles might start looking a little bit different. I think there's already several companies that are kind of calling everyone a data scientist now and blurring those lines. And so I think, you know, within that idea of blurred lines, it's important to know what you like and advocate for yourself to do what you like.

And then another piece of that, that's something I really like to speak about this when I'm interviewing candidates for our team, is to have an open conversation about, you know, like you said, I was seeking out this change from a traditional statistician to what looks a little bit more like a data scientist, you might say. But like fresh grads, to have a conversation about what that means and what that means in terms of your career. I always like to have that conversation because the industry as it stands right now has more of those traditional roles. And maybe it will change over time, but just kind of having an open conversation about it because it might be a new concept for people that are new to the industry.

Thank you. Yeah. So maybe at some point I'll pull all the answers to these this question together, but I know we've asked it in a few different Hangouts. Is there a piece of career advice that you've been given or that you've given others that stands out to you? Sure. So when I joined the CRO we've talked so much about back in 2017, I was just very lucky to be scooped up by a very passionate mentor who has 100% guided my career. And he's given me so much advice. And the advice is both solicited and unsolicited. And it ranges a big gamut. You know, there's statistics advice, there's programming advice, there's advice about my mortgage, there's advice about how to plan my wedding. And so it was hard to sift through many, many words of wisdom to answer the question. But I think, let me take a sip of water.

I think one thing that I didn't know at the time was going to be so important, but luckily was very kind of like lead by example, is how to be a remote employee that is engaged with a team. I didn't necessarily think I would be a remote employee. And that changed and most of us are now. And I think the single greatest advice is just pick up the phone, like to be a part of the team so that you're not siloed away, form personal relationships, and maybe not everyone wants to form personal relationships, pick up on that, if that's the case. But to call people that you work with and ask them a question about work and ask them a question about their personal life. I think that was one of one of the better lessons I've learned.

I think the single greatest advice is just pick up the phone, like to be a part of the team so that you're not siloed away, form personal relationships, and to call people that you work with and ask them a question about work and ask them a question about their personal life.

So much. Thank you all for spending your Thursday with us. And thank you, Stephanie, for sharing your experience and knowledge with all of us. This has been great. I love the engaging conversation and definitely one of the more lively chats as well. That's what I like to hear.

Bye all. Have a great rest of the day. Thank you so much.