Resources
Jan 8, 2026 |
55 min
|
184 views

Getting Started with LLMs in R and Python: Tools and Best Practices - Jan. 8, 2026

LLMs are changing how we write code, build tools, and analyze data. This talk is a practical introduction to working with LLM APIs in R and Python, including connecting to models, tool calling, and evaluation. We will also show how LLM-powered tools can support common data analysis tasks like exploratory data analysis, as well as briefly discuss privacy and security considerations when working with protected data.

Sara Altman is a Senior Developer Advocate on the AI Core team at Posit, where she develops technical content and educational resources on using LLMs and AI tools for data science in R and Python. Before joining the AI team, she helped build Posit Academy, Posit's educational platform, and taught data science and R at Stanford. She has an MS and BS in Symbolic Systems from Stanford.

image: thumbnail.jpg

Transcript#

This transcript was generated automatically and may contain errors.

Again, but it's great to be here talking to you all and hopefully we can chat a bit more interactively after I talk for a bit. So yeah, today I'm going to talk about getting started with LLMs in R and Python. I'll discuss various tools for doing that and then some best practices.

If you're already familiar with, you know, working with these APIs from R or Python, the beginning will be a bit of an intro, but then hopefully towards the end there will be some more information that's relevant to you. And if you haven't done anything like this in R or Python before, this should be hopefully a good intro. But yeah, like I said, please, you know, ask questions when we have time. I'm happy to answer anything.

Great. So to start, I'm going to talk about how to get started with Elmer and ChatList, two packages for working with LLMs from R and Python. Then I'm going to talk about how you might add new knowledge to LLMs with system prompts, talk about tool calling, which is a way to give LLMs new abilities. I'll briefly mention some AI tools for doing data analysis, and then we'll also talk about some privacy and security concerns at the end.

How LLM APIs work

Great. So you might already know this, but most LLMs are accessible through HTTP API. So this is just like sort of a normal API. If you've ever queried an API to get any kind of data, it's pretty much the same idea. And this means that you can interact with LLMs from languages like R and Python. So instead of, you know, using something like ChatGPT in the browser, you can interact with an LLM from R or Python or any programming language.

And there are a variety of packages for doing this. Posit has two that make it pretty easy to interact with LLM APIs from R and Python. For R, it's called Elmer, and for Python, it's called ChatList. And you don't really need to know much about how LLMs work to get started, but there is just a bit of setup, and this might differ depending on your specific organization and the models you have access to. But generally, you need an API key or some way to authenticate to these models. And if it's an API key, you're going to add that to your .R environment file if you're using R or a .N file, something similar if you're using Python, and then install the relevant package. And that's pretty much all you need to do to start interacting with an LLM from R or Python.

Basic chat in R and Python

Okay. So let's see what the code actually looks like. It's really only I'll show you R right now, and I'll show you Python in a second. There's really only three lines of code that you need to start chatting with an LLM from R. First is just loading the library, and then on line three, we have sort of the core capability in Elmer, which is this chat function that sets up a connection to an LLM. Here I'm setting it up to an Anthropic LLM. This is going to choose a recent cloud model by default. So you create that chat object. This is going to control our communication back and forth to the LLM. And then to send a request to it, we call the chat method on the chat object. And I'm going to just send it something. I asked it to summarize Picasso's style in five words. Let's see what it says before we move to the Python code.

So it gives us and it gives us back a response as a string. So with just a few lines of code, we've already set up this connection to the LLM, and it's given us back a response. This is what it looks like for Python. This is the ChatList code. You'll notice that it looks pretty similar to the R code. The Elmer and ChatList APIs are very similar.

What if we want to continue the conversation? Usually you don't want to just send one request to the LLM. You want to have a back and forth conversation. So now we call that same chat So now we call that same chat method on the chat object. We're sending another request off to the LLM. Let's say now Georgia O'Keefe. It remembers what we had said previously in the conversation. And so it gives us back a, you know, similarly formatted response. It's five words describing her style.

So importantly, every time you call chat on that chat object, the thing that's controlling the communication back and forth to the model, it is going to continue on the conversation. So it has the entire context of your previous conversation. And so you can do things like say now Georgia O'Keefe, and it knows the previous query that you sent. And so it's able to adapt its response. Again, the Python code looks very similar. It's a very similar idea.

If you want to switch providers or models, these packages make it very easy to do so. For LMR in R, you can pass the chat function as string with a model provider. And if you want to specify a specific model, you can use the model provider slash model syntax. So that's the second example here where it says OpenAI slash GPT5. We're requesting that a connection be created to GPT5, which is provided by OpenAI. In ChatList, you just use a specific function for each model provider. But it's a similar idea. So this means that with these packages, you can connect to a wide variety of models. And there's also functions for connecting to local models.

So these are models that you would download to your laptop or someone has downloaded to some server. And then you want to interact with that.

Getting started: recommendations

Okay. So this is our sort of normal advice to people who are getting started working with LLMs. I just want to say this might not apply to you depending on what kind of data you work with and the constraints of your organization, but I thought it would be useful to go over anyway. First is that we generally recommend that if you're starting to play around with these things or build something in R or Python that interacts with an LLM to start with one of the frontier models. Because like here I said best, but just the sort of latest models from these large AI labs, these tend to be the most advanced models available. And the reason that we say this is not because of like loyalty to these companies, but just because if you want to know what LLMs might possibly do or what you might be able to build with them, it's a good idea to start with something very powerful and then work your way down to something smaller if you're able to do that for your use case.

Local models, again, these are models that you would download to a computer and then so you're not doing any querying over the internet, are very enticing. They sound like they solve a lot of problems. Unfortunately, even the best ones now are not quite at the level of these paid models. So you may need to use a local model or you may want to. I would just note that like if you're just experimenting with things and not, you know, working with any proprietary or protected data, you might want to just start and see what the frontier models can do and then work your way, you know, towards cheaper or smaller or local models.

And then the other thing is if you're new to working with LLMs or like you're new to working with the APIs, it's often good to get a grasp of how these things work and then think about security and data privacy later. So like don't try them out, you know, first thing with a project where you're really worried about data leaking or data privacy or any of that. And we'll talk about privacy and security at the end as well. Yeah, so just like pick a project where you're not overly concerned about that, just so you can get a handle on how the APIs work and then you can move on to something maybe more realistic.

Adding knowledge with system prompts

Okay, so I showed you a basic chat, but let's move on to a slightly more advanced content. LLMs can do a lot, but they don't know everything. They're trained at a particular time and they have sort of a set of knowledge that is contained within the model. But you might have additional information that you want the model to know. This might be newer information, knowledge that only you or your organization possesses, real-time information, all of that. So one way to do this is by adding knowledge to the model.

In this way, you can give it context about a specific domain or data that it does not have.

Okay, so to do this, to tell the model something that they don't already know, our recommendation is to start with system prompt customization. This is relatively easy. Effective and pretty fast. And a system prompt is essentially a set of instructions from you to the model that controls its behavior. The system prompt will shape every response that the model gives in a particular chat or in a particular conversation. So you can put instructions or background knowledge in the system prompt. And in this way, you can exert quite a bit of control over how the model behaves. You might have heard of things like fine-tuning or rag. And these things are useful, but they're sort of more, they're techniques that take a bit of finessing. So our recommendation is to start with system prompt customization. And then if that doesn't fulfill all your needs, you can move on to other techniques. Let's take a look at what this looks like.

So for Elmer and ChatList, to provide a system prompt, you're just providing a string to the chat function. So here I'm telling it to behave as if it's a data analyst who answers very briefly. Looks similar for ChatList. And again, this system prompt will persist across the entire conversation.

In real life, you often will have system prompts that are very long. So our recommendation is generally to store them in markdown files. So you can put your entire prompt in one markdown file, store it across various files, anything that works best for your organization.

So here's one example. This bit of code at the top is just constructing a system prompt. One advantage of storing in markdown files is that you can have things like maybe you have a clinical trial protocol that lives in a document that you're using for other things as well. And now you can provide that to the LLM in the system prompt. So I'm just reading in the file and then pasting with a bit of text. And this is going to form a system prompt that contains the entire protocol and the instruction to answer questions about that protocol. And again, we pass that to the system prompt argument. So now this LLM has knowledge of our specific clinical trial protocol that it did not have before, which means we can ask it questions like, can we enroll some patient with particular characteristics? And it will give us an answer back that is informed by that system prompt. So even though this is essentially just a text file, it can be really powerful and you can really control the behavior of LLMs just with text, just with these system prompts, as well as give it new information.

So even though this is essentially just a text file, it can be really powerful and you can really control the behavior of LLMs just with text, just with these system prompts, as well as give it new information.

Okay. A few tips and tricks for working with system prompts. First is that it, just like if you were explaining to another human, you really want to clearly explain what you want it to do. The more explicit you can be, the better. Examples are also very useful. If you have a specific way you'd like the model to answer, provide examples of that. You can also add structure as well as use LLMs themselves to help you draft your prompts. CLAWD, for example, has a prompt generator that can help you write a system prompt.

Tool calling

Great. Okay. So we just talked about how to give LLMs new knowledge. What about giving them new abilities? This is where tool calling comes in. Tools give LLMs the ability to add new skills to their tool set. This lets them take action on the world, like, you know, affect your environment, make changes to files, run code, that kind of thing, and also gain access to real-time information.

Okay. So before I talk about tools, we're just going to talk a little bit about how LLMs actually work and clarify what they can and can't do. So on their own, can LLMs access the internet, run code, send an email, do any of this stuff? Let's talk a little bit about tools.

Let's see. So what if you just ask the LLM from Elmer, what's the weather like in Seattle? Implied is like right now, not, you know, four years ago or whenever the model was trained. So if I run this code in R, it tells me I don't have access to real-time weather data, so I can't tell you the current weather conditions in Seattle, Washington. This is because LLMs don't have access, you know, just in their core technology to real-time data. They were trained in a particular time on a particular set of information, and that's all they have access to, unless you have given them access in another way. Okay. What if we ask it to affect the world in some way? For example, what if we ask it to write a file to a CSV?

Again, it basically says it can't. It says it's not able to create or write files on a computer. It can only provide code and text responses. So they don't have the ability to access up-to-date information or the ability to affect the world. And this might seem a little off to you, because you have probably seen an LLM access real-time information or affect the world in some way, or, you know, have heard that they can basically give you access to real-time weather data.

Whatever you want them to. And the reason is that you can hook LLMs up to tools to give them these abilities, to give them the ability to access real-time information or affect the world in some way. Tools are essentially extra capabilities that you give the LLM.

Tool calling

And a tool at the core of it is essentially a function and some metadata. If you've ever written an R or Python package, it is kind of like if you wrote a function in a package and then documented it. That's all you really need to give the LLM to give it a new ability, is a function and then metadata telling it how to use that function.

So let's see what it looks like. I'm just going to go over this example in R, but it works very similarly in Python. And I'll give you the link to the documentation for Python after this. Okay, so let's go through this. First, we need to define the tool. This is a tool function from Elmer. And this is going to allow us to define a tool that we're going to hand to the LLM to give it a new ability. Again, I said a tool is a function plus documentation. So this line is the function. We're giving it a bit of code that it's going to request to be run. In this case, it's a function that queries a weather API for a particular latitude and longitude.

And then we give it the metadata. So a name for the tool, a description, and the arguments. And again, this is essentially like a function and documentation. We're telling the LLM how to use a tool. After that, we register the tool. This lets the LLM know it exists and that it can request to use it. Okay, so once we run this code, now we can ask the LLM, what's the weather in Seattle?

And it actually gives us a real response. So you'll notice that in the output, it shows that it did a tool call. It requested that the tool get weather be run for a particular latitude and longitude that the model chose. It received a bit of data back telling us the current weather in Seattle. And then it used that data to inform its response back to the user.

Okay, so importantly, with tool calls, the LLM itself is not running code. It's not that, you know, it took a bit of our code and then ran it, you know, on OpenAI or Anthropic servers or in the cloud or somewhere else. This code is running on your laptop or wherever you've requested or wherever you've set up this conversation. Because LLMs cannot run code by itself. They essentially just take in text and output text. But they can request that code be run.

And that's what's happening here. Instead, the LLM controls when the tool is called. So it decides when in the conversation it is appropriate to ask. It's appropriate to find out, you know, weather for a particular location. So it controls when the tool is called. And it controls how the tool is called. And this mostly means choosing the right arguments. You might have noticed in the previous slides, we didn't need to tell the model what latitude and longitude to use for Seattle. It knows that information. So it can choose the correct Latin long to pass to that tool. It's controlling how the tool is called.

But the code is still running on your laptop or on whatever computer you've set up this chat. And, again, tool calling Python works very similarly. Here's the link to the documentation for chat list tool calling.

LLM performance and evaluation

Okay. At this point or maybe previously or all the time, you might feel kind of apprehensive about LLMs. Maybe kind of like this cat encountering a Roomba. And maybe a little bit curious as well. You might have heard that LLMs can do cool things or very powerful things, but they also hallucinate, that they have privacy and security concerns, that they're wrong a lot, that they're untrustworthy, all of these things. So I'm going to talk a little bit about what LLMs are actually good for and maybe what you should avoid.

And the first thing is that LLMs are jagged. LLM performance is in some ways very unpredictable. You might think that LLM performance looks a bit like this plot where really easy tasks are very easy for the model to do. It performs great at them. But then as tasks get harder, model performance goes down. And really hard tasks are nearly impossible for LLMs.

But that's not really how it works. Instead, this plot probably looks a bit more like this. There are some easy tasks that models are terrible at, some easy tasks that models are great at, hard tasks that they're great at, hard tasks that they're bad at. For example, coding is often very difficult. Models can be very good at coding. However, they are very bad at counting. You might have seen the example of they don't know how many R's are in strawberry. If you give it a R vector, they're actually very bad at counting the number of items in that vector.

And so because of that, I generally encourage you to think empirically instead of theoretically about LLMs. It's okay to kind of treat them as black boxes where you don't really know how their internals work. And instead, try it out and try through experimentation, figure out if they're going to be good at a given task instead of trying to theorize, you know, from your armchair. Because you might be surprised both ways. There are things that you think they could almost surely not do that they can clearly do today. But there are also things that you think maybe like counting that surely they are good at and can do today that it turns out they're terrible at. In general, I'd say experimentation is much more useful when thinking about LLMs than intuition about how the model actually works.

In general, I'd say experimentation is much more useful when thinking about LLMs than intuition about how the model actually works.

To help you do that, there are R and Python packages that can help you build evaluations for LLMs. So these are essentially experiments that you can run that assess how well an LLM is doing a particular task. For R, there's an entire dedicated package called vitals. And then there's also built-in evaluation support in chat list for Python. So evaluations can help you compare models, compare prompts, ensure your tools are working as you want, all of this. It's very useful.

Along with my coworker, Simon Couch, we've been writing a series of blog posts looking at how well various LLMs generate R code using the vitals package. And I made this little app that you can see the link to here where you can compare how well various modern LLMs do on R code, do on R code, generate on like a particular R code generation task. So this is mostly, I'm mostly giving this to you as an example of vitals. You can see the code behind the app if you click on that link. Okay, so again, my recommendation is to not just guess if a model is good enough, but to test it on your actual use case and measure how well it does. And vitals and chat list can help you do this.

AI tools for data analysis

Okay, what about AI tools for actually doing data analysis? So far, we've been talking about packages that let you build the things that use LLMs from R and Python, but what about tools for actually carrying out data analysis?

Posit has built several AI powered tools for doing data science. I'm just going to briefly introduce them here. So the first is Positron Assistant. Positron, if you haven't heard of it, is Posit's next generation data science IDE. It's like a, you know, you might think of it's, if you've used RStudio, it's a similar idea where it's built for working with data, but it's designed for both R and Python. So Positron Assistant is built specifically for data science workflows. You can generate code, refactor code, debug, can answer questions for you.

And this is, so it's a coding assistant versus DataBot, which is the other agent that is built in Positron, which is specifically for doing exploratory data analysis. So DataBot can help you explore and visualize your data, doing in minutes what might have previously taken you hours. And it's also designed for really short feedback loops. So it doesn't, you know, work autonomously for 10 minutes and then come back to you with a summary of your data. Instead, it's more interactive. It does one thing, gives you the results and plots, asks you what you want to do next. You send a request off, it comes back. So it's a little bit more collaborative. And yeah, again, this is available in Positron. DataBot and Positron Assistant are really not designed to replace your judgment or your domain expertise or any of that, but just to accelerate your workflow.

Privacy and security considerations

Okay. At this point, you might be wondering, are tools and ones, are these tools and similar ones safe to use? Positron Assistant and DataBot and tools like them can execute arbitrary code and see your files. This is where they get their ability to do powerful things like explore your data, help you write code, all of that.

But this means that, you know, they can see your data, they can see your files, and they can run code that queries that data, moves your files around, all of that. And they can incorporate the information that they gain into queries sent to the LLM. So this means your data is going somewhere, and it is being viewed. But let's talk about exactly how that works for a bit.

Okay. So now I'm going to just briefly talk about privacy and security concerns. I know this is especially relevant if you're working with any kind of protected data, you're working with health data, you know, HIPAA data, any of that. First, I just want to briefly clarify the difference between a model provider and an LLM, because this turns out to be important when talking about privacy and security. So the model provider is the company that hosts your model that you might be paying, or like where you're getting your model from. So this is something like Anthropic or OpenAI. Or if you are using a local model, it might be something like HuggingFace. And this is in contrast to the LLM itself. This is the actual model that generates the responses to your queries.

Okay. This is important because LLMs themselves, the actual model, are stateless. They don't have any memory. They don't remember your prior requests. If I send some data over to an LLM, it is not storing that anywhere. It takes in that input, it generates a response, it sends that response back to me, but it does not store that information anywhere because of how the LLM itself works. They're stateless.

You might be saying, you know, I know that's not true. You can carry on a conversation with them and they'll tell you what they're doing. I know that's not true. You can carry on a conversation with them and they will remember my prior request. And this is because each time you send a request to the LLM, it gets the whole conversation. So if you've had 15 back and forths at the 16th request, it's sending all those 15 requests to the LLM. This happens each time. It's sending the entire content of the conversation. And you have to do that because of the statelessness of the LLM.

Okay. So this is good news for data privacy. However, your model provider may log and store your requests and use them for other purposes. So this is why I've tried to clarify the difference between these two.

Okay. To make it a little bit more explicit, I have a diagram. So if we're talking about a tool like DataBot or Positron Assistant that can see your data, it can see our Python session, it can see all your variables. Posit itself does not store your data anywhere if you're using our AI coding agents or any other time you're using a tool like Positron. But then DataBot or Positron Assistant sends a query that might incorporate that information to your model provider. And then the model provider sends that query to the LLM, but then might also do other things with it. It could use it for future model training, it might share it with third parties, and it might store your data. And all of these, those three things are in dotted lines, because those are maybes. Those aren't necessarily givens. And this is why it's very important for you or your organization to develop some kind of trust with whatever model provider you're using. So Posit doesn't store your data. I told you that the LLM itself doesn't store your data, but your provider might, depending on your or your organization's agreement. The good news is that zero data retention agreements, HIPAA compliant agreements, or other arrangements are relatively common for organizations to work

out with model providers. So you can have an arrangement where they absolutely do not store your data, share it in any way, or use it for future training. But these are generally not defaults. If you just sign up for a free ChatGPT account, they probably are doing something with your data, which is why it's important for your organization to have a specific agreement with the model provider. If you want to learn more, we have a blog post on this that goes into a bit more detail.

Okay, so I know I talked about a lot today. There are even more packages that we at Posit have developed for working with LLMs from R and Python. Here are a few of them, if you're curious. So we talked about Vitals, Elmer, and ChatList. There's also things like ShinyChat, which let you build AI-powered Shiny apps, Ragnar for doing RAG from R, and many others. I've also put a link to some of them here.

Just to help you out, if you want to click through these links, I'll put a link to my slides in a little bit. Along with my coworker, Simon, I also write a bi-weekly, meaning every other week, newsletter where we cover external AI news as well as internal Posit AI news. This is the link to all the editions that we've published so far.

Okay, so thanks. I'm looking forward to questions if you have them. This is the link to my slides, and here are some links where you can get in touch with me. Thanks.

Q&A: LLM performance for R vs. Python

Awesome. Sarah, thank you so much. We have a couple questions that have stacked up in the chat, so let me go through those first. There was a question from Tyler in Snohomish. Tyler, do you want to unmute and ask?

Yeah, happy to. Thanks again for the great presentation, Sarah. I was just wondering, when you were using the vitals package, were you able to assess, is there any kind of noticeable difference in LLM performance for code generation of R versus Python? Yeah.

Yeah, I think, so that screenshot of the app I showed, that was just for an R task. We also tried to look at how well they generate Python code, and if you look through those blog posts, you'll see one that's specifically on Python, where it looks like the models are performing much better than they do for R. I think you can't really compare the performance of Python and LLM in that way for a couple of reasons. Say, in general now, the LLMs are very good at generating R code as well as Python code. I think maybe six months ago or so, people often felt like they were better at Python. I think there's just more Python information on the internet in general, so the models, when they were trained, had more Python knowledge, but now they are very good at both. I think if you're trying to choose between them because you want help from an LLM or some kind of coding agent, I think it's pretty comparable at this point, although it might depend on your specific model, because some of them are better at different tasks or different languages.

That makes sense. Thanks so much. Cool. Yeah. Next question from Tina Lai in New York City. Tina, do you want to ask your question on LLMs?

Q&A: Stateless models and data storage

Sure. I have a pretty basic question. Having not worked with that many LLM models, my question was really, do all LLM models work the way you're describing where they don't store it, where every prompt kind of takes in your whole conversation and send it again, or are there other types of LLM models that are newer that aren't doing that kind of processing?

Yeah. My understanding is that the core LLM model, the kind of model that LLM specifies, is always a stateless model. I think you can add states to add on modules to the model, or by tweaking how the model works in some ways, but the core technology is a stateless model. The way that generates your response is that it just takes the information in and then generates a response, and it's not storing it anywhere. I think there's ways to augment the model to give them memory, but the core design of an LLM is that it is stateless. I think this is kind of confusing because now when we talk about LLMs, you're usually not just using an LLM that you downloaded to your computer and it's just a pure LLM. You're using ChatGPT or a model that is provided from Anthropic or something, and the model providers have memory and are storing your information, possibly. If you've had a bunch of conversations with ChatGPT, you might notice that it remembers things across conversations, and that's all because of something that OpenAI is doing. They are storing that information somewhere, and it's not like the LLM itself.

They are storing that information somewhere, and it's not like the LLM itself.

Yeah, I hope that answers your question.

Yeah. I guess it kind of leads to a follow-up of you were saying OpenAI might remember previous conversations. Is that something that we will empirically discover, or is that something they've actively announced in a transparent way? I think they say that they do that.

I think if your organization is working out a contract with OpenAI or Anthropic, you can have explicit terms where they will not store your data in any way. I think even if you're a normal consumer using ChatGPT in the browser, you can opt out of them using your data for training or remembering across conversations. I would say in general, be wary of what they're doing with your data. It will typically be up to your organization's security team or IT department, if you have it, to work out a specific agreement so that it is safe to use these models with protected data. Yeah, I think my take on that is just assume that the data are being in transit or being used somewhere unless somebody very clearly tells you that they are not. Yeah.

Q&A: Zero data sharing agreements and tribal data sovereignty

Victoria from Northwest Tribal Epicenter, did you have a question you wanted to ask? Oh, yeah. Thank you so much. Hi, Sarah. That was really super helpful. As people on this call may recall, my main issue is protecting tribal data sovereignty and tribal ownership of data. And so my question was, are there examples where model providers have entered into those zero data sharing agreements that you mentioned? And if so, are there some examples or is that something if I reached out to you online, is there an example kind of template that you might have access to that we could review?

Yeah. So this definitely happens. I think zero data retention agreements are relatively common for any organization that has a contract with these companies. And then you might have an even more restrictive HIPAA compliant or similar agreement. So as far as templates, I don't really know. If your organization has a contact with whatever model provider you are trying to use, they will know how these work. Or other organizations that do similar work to you may have recommendations. This is usually something where I work, this is something that our IT team, security team, and the executives work out with whatever model provider we're using. And then it's like a contract that you've agreed to. I'm sure templates exist or you can see an example of a specific agreement. And talking to someone at your desired model provider company, they'll probably have more information for you. Great. Thank you, Sarah. That's really super helpful.

Q&A: Training your own LLM

Great. Anyone else want to just pop your hand up or unmute to ask your questions, please? I have a question. Can you hear me okay? Thank you for the presentation. That really makes me wonder what's possible to develop in R or Python using these LLMs. It really helped expand my mind. I'm curious if you think, if somebody went out there on their own computer and tried to design their own, I don't even know if you call it an LLM. My understanding is that LLMs are trained using supercomputers and a huge amount of computing power. Can an individual compete with these big AI labs and trying to develop an LLM themselves, do you think? I'm just curious about your opinion.

I think, like, so, I think compete, no. Like, I think it's really hard to compete. No. Like, I think it's almost, it would be almost impossible at this point for, like, an individual to make an LLM that is as good as these state-of-the-art models. I mean, they just have, you know, the stores of data that they use to train these models are massive. They're using a massive amount of compute. And then they have, you know, lots of engineers working to tweak the model to make it good. But you can, if you just are curious about, like, training LLMs, you can do that on your own. I'll try to find the link in a second. But there's, I saw something where, like, you can train your own LLM for, like, $100 or something, which seems kind of like a lot. But I think relative to how much it costs to train these big models is probably relatively small. And supposedly, you can do it on your laptop. I think this would be mostly just if you are curious about how it would work to build your own, but probably wouldn't be that useful for, like, something that you wanted to do professionally, like using that LLM.

And then the other thing I would say is that there are local models. So, these are models that, like, there are open weights, meaning that, like, all the weights, which are the things that, like, the model's parameters, essentially, are open. So, you can download that model to your laptop or to some computer and then run it locally. So, you're not sending anything over the internet. You're not sending it to OpenAI servers or whatever. And you're not building this yourself, but you do have more control over it. So, technically, if you wanted to, you could fine tune a local model. Your organization can have its own local model that it uses. Like I said at the beginning, generally, these models are still not as good as the big paid models, but they're getting better, and some of them might fit your use case and are good at specific tasks. Okay. Thank you. To add to that, you could also play around with fine tuning a model, where you take an existing LLM that's created on this vast archive of data and then sort of adapt it to your particular task. So, that gives you something that focuses in a little bit, but contains that same processing power that you may want to carry with you into your effort. Yeah.

Q&A: Human in the loop for code review

Hey, this is Prachi here. I'm from New York State Department of Health. So, can the LLM explain the code logic for the code test generated? Because in New York State, we have this requirement that there should always be human in the loop. So, like for code review, when you're generating something, is there a prompt which works saying, like, generate the code and explain the code logic along with it?

Yeah, you can definitely ask them to explain any code that they generate. Are you talking about, like, if you were using some kind of, like, coding assistant where it's helping you write code? Yeah, yeah, exactly. Yeah, you can definitely ask them to explain any code that they generate. For something like Positron Assistant or DataBot, the two that I talked about, like, they don't by default explain every bit of code that they write, but you can ask them to do it. And both are designed so that there's a human in the loop, like, they're not fully autonomous. They ask you to allow them to edit files or run certain pieces of code, and they're designed for back and forth. So, you're reviewing the code or at least looking at it, then they do something else, and then you look at the new code they generated, and it's back and forth with you and the model. Yeah,

because the real decision is in New York State, we have to make sure that everything is human in the loop and everything is not autonomous, and we are able to explain that in English, like, in code, which is great, like, you know, what it is happening. So, we always need to use that way.

Yeah, yeah, that makes sense. Thanks.

Go ahead, Ian. Yeah, great presentation. I have a question of whether you've seen any use cases of people accessing LLMs in this way from R or Posit, especially anyone in government?

Yeah. So, yeah, I definitely have seen use cases. Common categories of things people do are, like, yeah, make internal applications to help people. So, you might make, like, an app that has access to all of your organization's documentation or something like that. So, employees can, like, chat with this app to access all documentation for, like, packages the organization has written.

You can also do things, like, make, actually, I think it would probably be easier if I just shared this. I'm going to share a link to a Shiny app that uses ShinyChat, which is a package that lets you build AI-powered Shiny apps, but what this is doing is it's letting you chat with an LLM in the sidebar, and then depending on what you ask the LLM, it updates the dashboard.

So, I've seen a lot of people making these kind of dashboards where, like, you have an idea of what you want the dashboard to show, but you also want dashboard users to be able to customize what they see. So, you can use an LLM to guide what is shown in the Shiny app. This uses a package called QueryChat. I get that, I think, and I've seen people get a lot of use out of that.

Yeah, I can think of, or does that answer your question?

Yeah, yeah. No, that's helpful. Thanks.

Cool. I think Jacob is next. Go ahead, Jacob.

Hi, Sarah. My question is, thinking about running, like, local large language models to simplify, you know, working with your IT department to get all these agreements in place, like, how many billions of parameters do you find the model needs to be to actually be decent at coding in R and Python? Because I know, you know, as you scale up, the spec requirements of your computer are also going to go up. So, I'm wondering, like, is there a sweet spot, you think, or is any good local model probably not going to run on most of our laptops well?

Yeah, I think it really depends on what exactly you're doing. This is another blog post we wrote recently. This is just looking at how good local models that are small enough to run on your laptop do as coding agents. So, this is not just across the board, but I think it gives a good sense of, like, so we tested 14 billion parameter models, a 24 billion parameter model, and a 20 billion parameter model. So, these are, like, I think that this is sort of the general size. It's, like, the maximum you might run on a laptop is, you know, 20 billion or so parameters. Bigger than that, you'd need to run on, like, a specialized server or something.

And if you look at this blog post, it looks like they do really badly. But this is just for, like, to be a coding agent. So, if you want to do something simpler, they might work great. And I think in general, like, they can generally continue getting better. So, like, the ones that are small enough to run on a laptop get better. So, like, if you wait a bit, it might be better.

But it really depends on your specific task. And you might be able to use something even smaller than that, depending on what you want to do with it.

So, I guess, yeah, I'm saying it depends. I would experiment with it. They're good at some things and then not good at others. And it depends on what you want to do.

Great. We're closing in on the top of the hour here. So, I don't know if you have availability to take a few more questions or if you have a hard time.

No, I'm happy to answer a few more questions.

Okay. I appreciate that. Moremi, go ahead.

Yeah, thank you very much. I was just wondering, so when you query through the Elmer package in R, is it pulling the paid version or the free version? Or if you have a paid version, do you have to link your account to that? How does that work?

Yeah. Yeah, that's a good question. So, yeah, you need some way to access the model. Like, Elmer is not going to, like, Elmer doesn't have, like, a subscription to ChatGPT or something that it can use for you. You have to have access to a model and then give Elmer the way to authenticate to that model. So, I think for, like, OpenAI models, Anthropic models or Google models, I think to get an API key for any of those, you have to have some kind of paid subscription. I'm not entirely sure about that. There might be some kind of free tier where you can have an API key. But I think you need some kind of paid subscription. And so then for that, you would have, they would provide you an API key, you would put that in your environment.

If you don't want to pay any of these providers anything, then you can use a local model. And so you would need to have some way of handling that connection. And I think on the Elmer website, it tells you how to set that up. But, like, you in some way need to provide Elmer with the information necessary to connect to a model.

But if he had an open source model, you could connect with that.

Yes. Yeah. If you have, yes, if you have downloaded it, you can connect to it.

Go ahead, Jamie.

Hi, Sarah. Thanks so much for being with us today. I think you covered part of my question just a second ago. But I was wondering, I've been using Claude an awful lot for SAS and R code generation. And it's been really handy. It's been really helpful. So Claude is performing really well in that area. How do you think Elmer would stack up for on code generation compared to Claude for R?

Yeah. So I agree. We've also found that the Claude models are particularly good in R code. So the good news is, like, Elmer is a way to connect to LLMs from R. It doesn't have a particular, it is not an LLM itself. It doesn't have a preference for particular LLMs. So you use Elmer to connect to Claude or whatever model you want. So if you want to do something with the Claude model from R, you would use Elmer to do that. So you can get, you know, all of Claude's capabilities through Elmer. It's like handling the communication to the model. It's not a model in and of itself.

So you use Elmer to connect to Claude or whatever model you want. So if you want to do something with the Claude model from R, you would use Elmer to do that. So you can get, you know, all of Claude's capabilities through Elmer. It's like handling the communication to the model. It's not a model in and of itself.

Right. So you would do your queries using Elmer in RStudio and then you could essentially not need to use Claude to do queries because you could do the queries right in RStudio. Is that, am I tracking? Do you mean like the Claude website, like going to the? Yeah. Instead of just going on the internet and typing into Claude. Yeah. I think it depends on what you're doing. So I wouldn't, I wouldn't just like randomly have a conversation probably with Claude from R. I would do that if I wanted to build like a script or an app or something in R that uses an LLM. But if you are just like are chatting to an LLM, you might still use the browser.

I think like, yeah, there's so many AI things and like ways to use the models. It can get kind of confusing. But I would say like you would use Elmer if you are building something that uses an LLM in R. I would use like a coding agent. So this is something like Claude code or positron assistant or data bot. If you want to help writing R code and you want it to like put that code in your environment for you.

And then you might use the website like you're, you know, chatting within the browser. If you're not really doing for general questions or like you want help with something, but you don't need it to put your code directly in your IDE for you. But I use like all three ways. It really depends on what exactly you're doing. Okay. That's helpful. Thank you. Yeah.

Great. Last call, anyone?