image: thumbnail.jpg
RStudio Team Demo | Build & Share Data Products Like The World’s Leading Companies
What you might not know is that we also make a professional platform, called RStudio Team.
Learn How RStudio Team Can...
- Help you scale your data science work
- Seamlessly manage open-source data science environments
- Automate repetitive tasks
- And, rapidly share key insights and data science products securely to your entire organization.
Timecodes
0:00 - Intro
4:18 - Hard truth of data science
10:22 - Serious Data Science
16:46 - Model management with R and Python
18:48 - Live Demo / RStudio Workbench
23:09 - RStudio support for Jupyter Notebooks
24:40 - Live Demo / RStudio Connect
28:01 - RStudio support for VS Code
30:05 - R and Python within RStudio
32:33 - Scale and share data science results
36:55 - Sharing previous versions of presentations
38:16 - Data Science team knowledge sharing
40:36 - Scheduling snd emailing data science content
43:55 - Live demo / RStudio Package Manager
48:09 - Data Science stories
49:37 - RStudio Team
52:59 - What makes RStudio different?
55:12 - Q/A - Learn More
Leading organizations like NASA, Janssen Pharmaceuticals, The World Health Organization, financial institutions, government agencies and insurance organizations around the globe use RStudio’s professional products to tackle world-changing problems and we’re inviting you to learn how. You’ll learn how RStudio Team gives professional data science teams superpowers, with all of the bells and whistles that enterprises need.
You can try RStudio Team free here: https://www.rstudio.com/products/team/evaluation2/
If you'd like to access presentation slides, sign up for future events, provide feedback and/or ask additional questions we've bundled everything together for you here: https://docs.google.com/document/d/1HGt7LSohhyxpCvETvVEFHugrdaSnTcZaXbI0jV5g9ok/edit?usp=sharing
Transcript#
This transcript was generated automatically and may contain errors.
Hey, everybody. Thanks for joining the RStudio YouTube Live for today. We're going to get started here in just a few minutes. I'm going to be going through quite a few different slides and we'll be having a lot of fun today. So we'll give some folks a couple of minutes.
We started maybe 30 seconds early, so give folks a little bit of time to roll in. But thank you so much for joining me today. My name is Tom Mock. I'm going to be your host for today. I'll be sharing some links and some comments in the chat. So if you do have any questions, feel free to post them there. Let me know how things are going and if you have any issues with the live stream.
Part of what I liked from last time was at the start, everyone kind of gave a shout out to where they're joining us from. You don't think of the exact location, but just kind of the city or even the state or the country you're joining us from.
All right. Give it a few more minutes. We're at the top of the hour for me here in Texas. It's about 10 a.m. So we'll give it a couple more minutes to kick off and then we'll head over and have a good old time. All right. So we've got a good amount of people in the room. I'm going to go ahead and start screen sharing my desktop and we'll jump into it.
So for today, we're going to be going over RStudio Team, which is one of the professional products and kind of the overall software suite that RStudio provides. You probably know RStudio from the self-named RStudio IDE or integrated development environment. And that's one of the core things we produce in addition to all the free open source software that we provide. Now we have to make money to kind of give away a lot of the things we create. So we also sell professional products that enhance some of the open source work that we produce and give away.
And so that's what we'll be talking about a little bit today as well as how the open source work can integrate inside of it. So I'm going to share my desktop real quick and we're going to go a little bit off script to start. You may be familiar with what is the RStudio IDE. So this is, you know, just looks like any old RStudio IDE session.
The benefit being that kind of for what I'm doing today, I'm actually running an RStudio workbench. So you'll notice that I'm working through a browser as opposed to my desktop.
And a couple other things that I'll be covering today. Now, while I could walk through for an hour and kind of just show you the IDE, I do want to have some slides to work on. So let's knit some slides so we can create a quick Schrodingen presentation that I can actually talk about for today. So here are our slides. We'll let those load up for one second.
All right. Well, that's great. So we got RStudio team, but I don't want to share from this browser. I'd rather actually share it from someplace where y'all can actually see it as well. So I'll take this file and let's confirm that I'm working on the latest one. Okay. So this was created about 20 seconds ago. Great.
Let's take that file and we're just going to deploy it to RStudio Connect. So in about five seconds, I went from working in my environment to having this hosted on RStudio Connect. And now I have a URL that anyone can access anywhere in the world. So these slides are now up at Colorado.RStudio.com. So now you can go actually look at these slides and follow along with me. So let's go to that. I'll go to the actual slides for today and we'll jump into that.
If someone from the other RStudio team could drop that link in the chat, that'd be fantastic.
So RStudio team, a single home for R and Python for all the data science teams. Again, I'll have the slides linked at the very bottom and you can walk through all of that.
The hard truth of data science
Part of what we're covering today is the hard truth that data science is really hard and it can be challenging to kind of get value from some of the work you're doing or get, you know, executive buy-in or business value from what you're doing. So in many cases, many data science teams actually fail to live up to what they promise or what they want to deliver because there's these challenges or these kind of momentum that they have to fight against.
So what we've done with RStudio team is try and solve some of these problems. And first, let's kind of talk about the problems or the barriers that data science teams might face, and then we'll talk about how we can work to solve those. So first off, data science teams find it difficult to create insights or impact decision-making or find it difficult to create, maintain, and improve their data products, their models, their APIs, their applications over time.
There's a lot of different ways that this shows up in terms of it could be difficult to recruit data scientists because you're using a very specific proprietary tool or you don't have all your data connected. For decision-making, maybe your stakeholders just don't understand the insights or there's a long, you know, duration between what if this happens or what if this happens and the actual answer to that question. It's also often difficult for teams to build on previous work for new use cases. They basically have to reinvent the wheel every time rather than kind of building off previous work.
Also, they might have analysis constraints or delays imposed by some of those proprietary tools. This can lead to slow iteration, which hinders alignment between the business unit and the data science team, or even irrelevant or outdated insights because things have been slowed down. Additionally, this can lead to insights become obsolete or difficult to reproduce. Again, the reproducibility kind of tenet of RStudio is true for both academia and for business settings.
And then lastly, siloed teams often lead to redundant work and there's lack of collaboration with too much time just spent like maintaining the tools or the difficulty of maintaining data science environments. Lastly, you know, teams can't often self-deploy. So they have to like go ask IT, can you take this and run with it or hand it off to a new team? And there's this disconnect between what you're actually creating and getting it into decision-making or getting it into production. And this is kind of lastly pointed out by just too much manual effort. You know, you're not able to rerun your analysis. You're not able to maintain the, you know, the work that you're creating.
Serious data science: RStudio's approach
So part of what we try and approach this from is here at RStudio, we try and solve these problems in terms of creating insights, impact and decision-making, maintain and improve over time things that are difficult by focusing on our three kind of core tenets. Open source work, a code-first mentality, and centralized or cloud-based kind of data science environments and production.
So open source basically means widely used. It eases recruiting, retention and training. You have an entire massive community in R and Python creating content that you can learn from, as well as libraries and different code examples that you can adapt. Code-first meaning it's flexible. You can actually craft it into exactly what you want. There's not this black box mentality where you just kind of plug something in and get something back. You understand the process. And then centralized or cloud-based by reducing the necessary work and enhancing collaboration, again, I'm working from a browser so I can access my work from anywhere and collaborate with my colleagues even though we're all remote in this time.
Open source also means things like comprehensive. So based on community contributions, there's tens of thousands of R packages on CRAN and tens of thousands of Python libraries across PyPI or other locations. Code-first also means you can iterate quickly and update quickly and you can adapt the code and understand by just changing small portions. And then for centralized work, your deployment provides stakeholders self-service access. So you can actually say here's the data product I've created or the model that we're using and have them actually interact with that as well as have it work against, say, your website in production.
And then lastly, open source means interoperable. So you can break down some of these analytic silos. You're not stuck with a specific vendor. You can move between clouds or even move between products because you own the kind of data science code that you've written there in terms of you're able to adapt it. Code-first meaning reusable and fully extensible. So again, you can diff it. You can put into version control and check all your changes and see what did we do two years ago versus what we're doing today. And lastly, package management is something that we're very keen on to support, again, reproducibility and easing the administration of open source data science in your enterprise.
So kind of summarizing this down, we've talked a little bit about these different slides. This serious data science concept is summarized as open source, code-first, centralized computation, or a cloud environment. The leading languages that we're focused on is R and Python. That's what we see the most of the data science teams using today, and that's where we've built a lot of our tooling around.
This serious data science concept is summarized as open source, code-first, centralized computation, or a cloud environment.
Introducing RStudio Team
And that's where RStudio team comes into play. So RStudio team is a suite of all three of our professional products. RStudio Workbench, which is basically the RStudio IDE running in a server, as well as VS Code and Jupyter for the Python users. RStudio Connect for sharing insights with decision makers via web apps, emails, APIs that are working production or other different assets. And then RStudio Package Manager, which is really the backbone between those two products, controlling, managing packages that data science need to create and share their insights.
Basically making it where even though you have all these different libraries you're working with, you're not putting this burden on IT to figure out how to manage that. You as the data scientist can kind of control your own environment as you go along.
So let's talk about this a little bit more kind of practically. We've talked about bullet points, but let's actually show what this process looks like a little bit.
What we often see is this massive chasm or this gap between the hard work that the data science teams are doing and the actual outcomes of that work. So either influencing decision makers or informing them or making decisions, as well as automated decisions or things working, models in production, things working on your website, things making decisions automatically within your organization or informing your organization. So let's talk about how we can kind of bridge this gap.
So data science teams are creating some type of insights or sharing insights to impact decision making, whether this is again, interactive web applications, reports, or APIs or other assets that are making automated decision through machine learning.
These data science teams are probably wanting to use a data science workbench where they can choose to work in their primary language. Maybe it's R, maybe it's Python, maybe it's both. They want to have that flexibility of having all this available, ready to go in one environment. So they're building things in R and Python. These assets need to live somewhere. There's tailored applications that actually have a front end so the decision makers can go and interact with it. There's reports that need to go out kind of automatically.
There's APIs that kind of, again, can work against the website or other, you know, software languages like Java or Scala or anything else. So these need to be published to a deployment server, basically somewhere to host all these data science products you're interacting with or creating. Things that exist on this deployment server can then be delivered, again, by web apps and emails and reports that inform decision makers when they need it. And they can actually have self-service applications so they can kind of understand what you're doing in your data science role. But also having things like APIs that are working in production and integrate into automated decision making in your website or against other software inside your company.
Now, the last part of this puzzle, you know, we've kind of made our way over to influencing decisions, whether true human-to-human interaction or automated, is package management in terms of R and Python or open source libraries, which is their strength. But you also need to kind of manage those different environments because the open source landscape is changing over time. So package management is important for easing maintenance as well as reproducibility of your analyses over time.
Now, if we overlay exactly kind of what we provide here at our studio, this is how it fits into the world. We do have a workbench environment in RStudio Workbench that allows you to analyze data in R and in Python. We have RStudio Connect that can host all the different R products as well as many Python products. So APIs in R and in Python via Plumber and Flask. Pins for datasets in R. Shiny applications with interactive front ends. R Markdown for reports, websites, or even entire emails and other things. As well as Jupyter, Streamlet, Dash, FastAPI, Bokeh, and Flask. So this rich kind of ability to publish many different data science products in R and Python.
I did see a question come in in the chat about Stata inside Jupyter Notebooks. That is something that in terms of R Markdown can actually support Stata chunks, other things. We're primarily focused on R and Python, but many of the things that you use in Jupyter Notebooks, whether it's extensions or other things, can work inside RStudio Workbench as well. But we're primarily focused on R and Python.
A real-life end-to-end example
So that's kind of, you know, again, a more general example. Let's dive real quickly into a real life example. So this is an example that lives at solutions.rstudio.com, BytePredict. And this is basically an end-to-end example of the exact workflow I showed before. But we've now replaced all those little images with actual data science products.
So the first part of your data science workflow might be data import, cleaning, fitting a model, training a model, you know, all the different things that you're doing in your day-to-day work. This can be done inside R Markdown. And with RStudio Connect, you can host that R Markdown so you can reference in the future or even schedule it to be executed on some type of schedule. So you might do nightly model training or batch training that goes against a database and then saves back the model somewhere. In our example, we will be pinning a model as a pin here. I'll be going over some demos here in a second. I'm just kind of going through some of the highlights above.
You can also save existing data sets. So say you do have a subset that you want to pull into an application. You can also extract specific data sets and save them as a pin or a data set living on RStudio Connect. RStudio Connect can connect to your databases. So you can have a live database connection against either a production database or one specifically tailored just for your web applications. The models will be served up as an API. So your model is trained, it's saved, and then it's actually put into production as an API via the Plumber package.
So this will then serve both a front end that's interactive in terms of your business users or your decision makers can go to a Shiny application and interact with it and see how does the model work and they can change different parameters and see how it changes. But it's also working in production in terms of doing the predictions for you on your site.
A couple other questions coming in in terms of where the slides are available. The slides are available at this link here at the bottom, colorado.rstudio.com. Or you can find the slides on my GitHub at rstudio-team-demo. And then lastly, you see RStudio Package Manager over here supporting all the different libraries. You see that there's at least five libraries in here, but in reality you have many other R libraries that are existing in this example.
Python support and open-source workflows
Now, there's some questions in the chat about Python, you know, in terms of, like, I want to do web apps. I'm not as big of an R user. I'm more comfortable in Python or vice versa. We also support a similar workflow or kind of this model management with Python and RStudio.
So here, rather than using R Markdown, maybe you're using a Jupyter Notebook. And rather than using a Plumber API to host your model in production, you're using Flask. And then to consume the model, your API consumers can actually query the Flask API running on Connect. And then your business users can interact with the Flask API with Plotly Dash, which is another dashboard software or another kind of interactive web application that we support.
Now, this is still pretty high level in terms of we're talking about mostly pictures and different things. But just the goal here is that we're showing you both R and Python are supported inside this environment. Yes, it can still interact with other different softwares in terms of maybe you're using a specific database, like you're using Snowflake or Postgres or you have a Spark cluster or Databricks. You can interact with those different things through the software as well. Because you're using open source data science, you are able to actually use anything you can do in R and Python. You're able to leverage within the RStudio team stack.
So as long as you have an ODBC driver or a connection you can make, you can connect to all these different data sources. Now, version control is another big thing in here. So if we go back, like all these different things, you may not want to put those all up in the public. So you could actually source different things from a private internal R package, for example. So maybe you write an internal R package that you don't want to publish on CRAN because it's got internal code or it's querying its internal databases. You can actually publish that package to RStudio Package Manager and never have to expose it to the outside world if you wanted to.
You can also use things like Azure DevOps or Bitbucket for offline or private or firewalled GitHub integration or whatever you want to do. All the different kind of resources are available there.
Live demo: RStudio Workbench
All right. So I know people are itching to actually see some more live demos or some coding. So I know I started at the beginning by jumping into RStudio Workbench and showing you a quick example. So let's go back there and have a little bit of fun.
So the kind of the takeaway from the initial part of the presentation was I can take these slides that I needed to rebuild and send them essentially instantly to RStudio Connect. So that ability to quickly iterate is very powerful. You might notice that in this RStudio environment, I've got a couple new things that aren't available in RStudio Desktop because I'm running in the cloud and because I'm running RStudio Workbench as opposed to the open source desktop version.
Number one, I'm authenticated. So for your IT team or for security reasons, I'm authenticated through single sign-on. So I have enterprise-grade authentication, basically making sure that any of the data that I'm working with is secure and that all the different proprietary things I'm working on are protected. I also have the ability to change my version of R. So I can switch from R version 4.0 to the latest at 4.1.1. All sorts of great things you can do in one session.
The other benefit is because I'm working in the cloud or I'm working outside of my desktop is I'm no longer tied to a specific kind of compute environment. So sure, I've got a powerful laptop, but my cloud environment is almost always going to be much, much more powerful.
So sure, I've got a powerful laptop, but my cloud environment is almost always going to be much, much more powerful.
So specifically, you might have, say, like an AWS instance or an Azure or, you know, Google Cloud, whatever different kind of cloud provider you want to use is supported, and you can actually have support for things like Kubernetes for additional scaling. You may not be familiar with Kubernetes, and that's okay. It's not required. What that basically means is that I can actually, from within my session, run an additional background script against the Kubernetes cluster and have that scale up as needed. So this option, be it a launch or job, actually allows me to specify the amount of compute I want to do for this background job I'm running. So I can run it for a demo server or limited to three CPUs, but this could be much larger in a production environment, and then I can change the amount of RAM that's available as well.
Now, you might think that's kind of silly. It's like, okay, I don't want to run an upload script as a background job, but what about some grid or model tuning? So this is a quick example of an end-to-end kind of long grid search that's going across a bunch of different hyperparameters, fitting a model and saving it out. Of course, I could run this in my existing environment, but that's going to lock up my console. I'm not able to kind of continue working. So what I can do with RStudio Workbench is start a launcher job.
It automatically pulls in the script I'm working on. So this tuning script that's probably going to run for a while. I'll set it to the maximum of three CPUs and eight gigs of memory, and then I'll start that. And now this is going to run in a background environment, and it's going to return back examples as it goes along. I'm still able to use my primary environment, so I can still do my math or do any exploratory data analysis or plotting, but I have this launcher job that's going to background and actually fail because my code had an issue in it in terms of the model fail. But it would run as a background job in this case if I had actually written the code correctly.
You can also do local jobs. So if you didn't have access to a remote Kubernetes cluster, you could actually run it as a local job in terms of running it on the same environment, but not having to exist on your small cluster. You could do it on a bigger one.
Jupyter Notebook and VS Code inside RStudio Workbench
All right. So let's take one step back real quick. So I've been focusing mostly on RStudio Workbench. RStudio Workbench might look a little bit different than your RStudio environment. So I started in RStudio so we could be comfortable, and now we're going to back a little bit out.
So RStudio team, RStudio IDE here, this is where I just was. So I backed out to the kind of home page or landing page, which has a few different projects I'm working on, as well as Jupyter Notebook and a VS Code session. Again, because we're trying to support both R and Python workflows or a mix of those workflows, we also have support for things like Jupyter Notebooks. So I can open up a Jupyter Notebook inside RStudio Workbench with the exact same authentication. I still have my home directory that is embedded here. So all the files that are available to me in RStudio and R are also available to me in Jupyter or VS Code. So I can open up the team demo. There's a Jupyter Notebook here. I can open up that Jupyter Notebook.
So let's let that load real quick. Again, I didn't have to set anything else up. I just am working from inside RStudio Workbench. I now have a Jupyter Notebook, and I'm running Python code in here. So here is the output from just reading in a quick data set and getting a summary of the different columns. I can create a quick graphic in Matplotlib and look at that. And then I can get the raw data as a table here, all this in kind of an interactive notebook that I wanted to use, all behind my authentication.
Now, when I mentioned authentication, I said single sign-on, but we support a lot more than that. We also support things like LDAP or SAML, other different Active Directory or Azure Active Directory support. So again, most of the enterprise-grade authentication you might use in your organization are supported here. And I just have to log in one time and then have access to all these different environments.
I can also take this Jupyter Notebook and publish it to RStudio Connect. So I can take the document. I'm going to include a couple other things. I'll include the R Markdown HTML and the zipped data and zoom out just a hair, and then I'll publish that to run. And so we'll publish the notebook real quick over there, and then that's going to actually live and it's going to deploy the notebook onto RStudio Connect.
So importantly, you can publish the finished document or the source document. So both in R Markdown and in Jupyter, you can publish these documents and actually have them rerun on a schedule or just post the finished document so that you can actually interact with them and rerun them in the future. So if I go to RStudio Connect, that's where all my content is going to live. So we can actually let that pop up in a second. We've got a lot of cool content in RStudio Connect, so it'll take a little bit to load, and we can look at the content I own. So right here, this is the demo or the actual slides I was using for today, and then we can see the Jupyter Notebook that I was actually working on published to RStudio Connect as well. And again, very quickly and just under a minute, I was able to get the notebook I was working on interactively onto RStudio Connect, and now I can share it with specific users in my enterprise.
Access control and sharing on RStudio Connect
While we're here inside RStudio Connect and looking at a notebook, you'll notice that it defaults to opening up the access portal. So with regards to authentication, this basically means once I'm authenticated in, I can do things, but I also want to open up access to specific users or to specific, you know, within my entire company. So it defaults to basically only I can see the content that I've published, which is a good idea. Like maybe you don't want to release it to the whole company because it has sensitive data. I can add specific people, like maybe I want to add my colleague Alex who can see this document because I need his review on it, or I need the entire solutions team to look at this and give me feedback on the document. RStudio Connect, because it's plugged into your enterprise-grade auth, is able to, you know, pull in these existing groups so you don't have to do the work as a data scientist of figuring out how does authentication work. That's baked in, and when you set up the software and set up the server, it'll actually integrate with your existing authentication.
Of course, if you just wanted to have it available to anyone in the org, you could always save it and make it where it's available to all users, login required. Basically, as long as they authenticate, they can access it. Or for what I did with the slides so that y'all could actually see it, I made it where anyone in the world could see it. So I made it anyone can see it, no login required. And it's got some more info, so I can look at, like, the screenshot of it. I can see, okay, about 100 people have viewed it today, so that's great, and a lot of good interest in it so far.
So we showed a quick Jupyter Notebook. It's interactive. We published it to Connect, and we were able to kind of control the authentication around it. But let's go one step further. So let's say that we want to, you know, we're not wanting to work inside Jupyter Notebook. We actually want to work in, say, VS Code, as we're a Python developer, and we want kind of a full-blown IDE as opposed to a specific notebook.
So I've logged into VS Code. Again, it's using my existing authentication. So I've logged into RStudio Workbench, and then I can hop directly into VS Code. It's got, again, my whole home directory. So all the files that are available in R are available in VS Code here as well. I've opened up that same Jupyter Notebook, so we're going to stick with the Jupyter Notebook example because they've got a nice interaction inside VS Code. But let's clear all the cells real quick. So we cleared all the cells, and then we can run, let's trust this one real quick, and we can actually run the code in each of these and interact with these cells inside VS Code.
While that's loading and connecting the different kernel, it's connecting to Python 3.7.5. It's basically making a REPL, or a read-eval print loop, basically saying, like, I can interact with Python live. So it has to do that for the first time when we set up the environment. Once that's set up, it'll go pretty quickly. So we get the same table, and I can always say, well, rather than a year, let's make it 365 days and rerun that, and now it says 52-week, 365 days. We can do our plot, and we get our matplotlib plot, and we can print out the raw data as well here. So still the ability to do that within VS Code.
And because we've embedded things like Jupyter and JupyterLab and VS Code inside of here, you can fully customize it. So you can extend it with the extensions you'd like, or other kind of integrations that you want to show. And I can always go back to RStudio if I want to get out of here and go back to R. I can go back to my homepage and go directly back into RStudio team, and I'm ready to go back here. All right. So still within RStudio, and we've
published some Python products, we've published some R products. Let's show a quick example of using R and Python inside RStudio. So you may not know this, but you can actually use R and Python chunks inside RStudio. So this will actually use a library called reticulate, which is a kind of a play on words for reticulated Python. This allows you to call Python code from R. So I'll run this. It's got my Python environment running. And then inside of our Markdown notebook, I can get the similar output to what we are seeing inside our Jupyter notebook. We have the exact same kind of table that we're doing with year. And again, I can change this from year to 365 days and re-execute the whole thing. Get that. So now it's 365 days.
I'll clear the plot so we can recreate it. We can create a plot in line, and we can get the raw data as a table. Inside RStudio, you can actually see both R and Python objects. So the R is just showing the connection to a Postgres database, but Python actually has the prices data frame, and you can explore that here.
Now, this might be a little bit kind of overwhelming. It's like all these different things going on. Again, RStudio Workbench is providing this front end to all these different environments. So RStudio Workbench provides RStudio, the ID you know and love and that you're very comfortable in. If you're a Python user, maybe you prefer Jupyter notebooks or VS code, and that's fine. We support those as well. So all of this is done within RStudio Workbench. Now, the data products you create in terms of R Markdown reports, Jupyter notebooks, Flask APIs, Plumber APIs, Shiny apps, Dash apps, Bokeh, Streamlit, all these different things you're creating in R and Python, those all get published to RStudio Connect. So you're doing all of your coding initially in Workbench. Once you've created something, then you host and publish it on RStudio Connect.
Touring RStudio Connect
So we've shown the different parts of RStudio team in terms of the IDE, the notebooks, and then VS code. And we've shown a little bit of this running in RStudio Connect. So let's close down a few of these things so we're not quite as overwhelmed. We've got that up. We can close the VS code environment. And then we've got a couple other different things here. So let's open up my portfolio dashboard. So this is actually a shiny application running on RStudio Connect.
I can show publishing this again in a second, but we've already seen a bit of the publishing workflow. So this might look like most shiny apps you've seen. It's got some different parameters on the side you can look at. It's got some built-in interactivity through, I believe, the dy-graphs R package. Or no, this is actually Plotly. So you can actually see the hover on different bars. And then you can change parameters, and the graphics will update in place based on the changes you're making to Shiny. So that all makes sense. Again, we have access to specific users or changing who can actually view these applications.
But with all the interactive web apps, so if you think of like Shiny in R or Dash and Streamlet in Python, those are actually server-based products. So they actually have a R or a Python runtime behind it.
So what RStudio Connect provides is, as a data scientist, you probably don't want to worry, you know, figure out how do I run a Linux box? How do I scale things automatically? How do I set up, you know, parallel work or multiple R or Python processes? Like that's not necessarily where you want to spend your time. That's what RStudio Connect does for you. So it provides the ability to scale up the, you know, scale up the actual server or the actual compute environment as more users come to visit it. So if we read this, this basically says this Shiny application can spin up three separate R processes that can each serve 20 connections. So this Shiny application just out of the gate can serve 60 people. Now I could change these parameters and make it serve 100 or 1,000, or in some cases we have examples of like tens of thousands of users that could be potentially supported. That's going to be based around how big your server is, but you as a data scientist don't have to worry about how does this actually scale in terms of figuring out all the different components. You have the ability to just publish your Shiny application and then change in these parameters if you need to.
you as a data scientist don't have to worry about how does this actually scale in terms of figuring out all the different components. You have the ability to just publish your Shiny application and then change in these parameters if you need to.
You can also set things like the timeout. So some Shiny applications or Dash applications run for a little while. So you can actually delay the timeout so that, you know, it doesn't kick off in the middle of something that's executing. But for most of the time, these are very quick in terms of you saw like I can change something and in like a second or two, it's going to change the actual component because it's going to rerun the R code or the Python code. And this is supported for both R and Python content.
So that was Shiny. We can do the exact same workflows for Python-based content as well. So let's go to a quick Dash application and let's do this Dash application for my colleague David.
So this is just one of the examples from the Dash team in terms of like their hello world examples, but a rich one that they have. And again, I have the ability to change these parameters as I see fit. So as a data scientist, I can publish my application. I can change different things and it makes the graphic change on its own. But I don't have to worry about how does Linux work or how does authentication work or how does scaling of an environment work. I just get to create the data science content that I want to put out there.
So we've talked a bit about Shiny and Dash. You do similar things with Streamlet. So let's close down a few of these little ones. Here's a Streamlet example. Streamlet's very similar to like a Shiny-based R Markdown in terms of they're kind of like a notebook style setup or a linear flow, but the same idea in that you change a parameter and then the code is re-executed to regenerate something. In this case, showing an output or a graphic, but this is running Python on the back end as well.
You can create something similar to this in R Markdown with a Shiny runtime that's also supported on RStudio Connect. And again, you have full access to the access controls as well as the runtime for changing the parameters of how much it scales.
Versioning and scheduling
The other benefit in terms of reproducibility is let's see if, let's go back to one of mine. So I have a presentation on production databases. We can look that I actually have multiple versions of this presentation. So I publish an older one at the very end of September and actually go back to a previous version. So I can maintain or I can overwrite. So let's say I have a presentation that I'm building or R Markdown report or a Jupyter report I'm building. Connect will handle the history of that report in terms of showing you like, oh, I can look at older versions of the same thing. So I can always look at the latest if I wanted to, but I could roll back to an older version if I wanted to show that to someone else. Be like, what were things looking like a month ago or six months ago or the last time I ran a report? And all of that is kind of built into Connect and you have the ability to make it where it doesn't keep any of the history if it's going to be really big and you don't want to store all those or you could store many, many versions of the same report over and over.
All right. So let's close down some more of these little guys. This is another dash application running on Connect. Another one you might worry about is within a data science team, maybe you're not creating as much content that's consumed by business users. And that's okay. Like in terms of like you're creating tooling for your internal team or you're creating knowledge repositories or knowledge sharing within your data science team.
This is an R Markdown report hosted on RStudio Connect that aggregates multiple data science things within an entire project. So if we scroll down, this is that end-to-end workflow I showed in the slides from earlier, which is bring some data in, creating some datasets that are saved to RStudio Connect as a pin, taking a model and saving it onto RStudio Connect and then serving that model up as an API that's interacting with Shiny applications or a website. So this aggregates all those different components. So as a data scientist, sure, I'm probably going to have some of this on version control and that's where I'm going to be like actually writing code. But if I want to just go interact with the things in production, this shows me all the different components of this very complex data science task. So the ETL scripts, you know, these are scheduled to re-execute every, you know, every 15 minutes or every day. So I need to check to see if those are going well and what the output was. There's modeling in terms of the scoring of the model and how well it did or the training of the XGBoost model. And then it actually can send out an email report if something goes awry.
So RStudio Connect has support for email servers. So if a parameter is met or is not met, it can actually send you an email and inline in that email embed ggplot or a table or attach a CSV or attach a presentation or attach a report. Whatever you want to do in R, you can kind of build that into a presentation and send it along.
It then goes along and has the different data sets that this is using. So it's pinning from a database into these kind of extracted data sets. It's got a link out to the specific API, link out to the specific API, the performance app, which is intended for the data scientist to interact with the API, and then the kind of production, you know, business user focused API interaction so that they can actually click on it with Shiny or with Dash.
So a lot of cool things going on here. Let's dive real quick though in terms of one of the things that resonates for me and what I use a lot is scheduling of content. So let's actually, we'll go back and go to the ETL. I often have tasks where, you know, it's something I have to do all the time and I want it to rerun on a schedule. I don't want to have to manually pull from the database, click run, and then publish to connect. Like that sounds like a lot of me clicking a lot of buttons. So what I can do instead is for this portion in terms of it's connecting to RStudio Connect and the database, it's cleaning the data and then writing out the data. I can actually schedule this to run every single day. So built into Connect, I can say schedule this output, rerun this exact code at this time every day, and publish the output.
So it's going to go through, and if we look at the history, there's many, many different versions of this report because it's been running every single day. So this report runs every day, does some type of cleaning or automate some type of model training step that I don't have to do manually. So now I can go spend my time on other things that are more useful of my time rather than clicking run every day or rerunning something manually every day. And importantly, because it's on RStudio Connect, the package environment is stable. So while my dev environment, I can install new packages and I can change my version of R, once I publish something to RStudio Connect, it's stable in terms of it's living there. It's got its own little environment that's controlled with a very specific version of R, very specific version of all these different packages that is static and it's controlled. So this report has been running for, I think, over a year. And I have other reports that have been running on RStudio Connect since I've joined RStudio over three years ago. So it's nice to be able to put something in production and just let it run, even though my dev environment where I'm writing new code has changed or the actual outside world has changed a bit.
it's nice to be able to put something in production and just let it run, even though my dev environment where I'm writing new code has changed or the actual outside world has changed a bit.
As far as the emails that get sent out, so you can see there's an email button here. There's an email that's sent out every time this is executed and it's sent to the person who published it. So in this case, Alex gets a report every day saying, hey, this ran successfully and here's the output. But it can also send it to all of my collaborators. So anyone who's also working on this package or on this project can get a copy of the report. I can click manually send this in terms of email report to me right now, even though the thing hasn't been scheduled right now, send it to me right now. And you can control basically how it shows up. With the latest release of RStudio Connect, you can kind of control saying like this is coming from RStudio Connect or sending it to whoever you want inside your organization. So again, your authentication is talking to that email server. And as long as your people exist in your authentication database, you can send these emails to them.
RStudio Package Manager
All right. So we're getting close to the 1043. So I want to make sure that we cover the last component. So a lot of cool things I could talk about RStudio Connect basically all day, but I do want to talk a little bit about RStudio Package Manager because it's kind of the unsung hero behind all of this. So the amazing part of the R and Python ecosystems is not only do they have an amazing kind of core language and base packaging available within them, but the package ecosystem that extends the capabilities. So again, there's tens of thousands of packages available, all of which can have many different versions. So from an IT perspective, they get nervous when they hear open source sometimes because they're like, oh, you know, I can't control it. How do I manage this? Package Manager allows you to store an entire copy of CRAN behind your firewall air gapped on your environment. So let's say for ggplot, for example, which I pulled up here, our version of Package Manager internally that we're using has the latest version as well as the archive versions going back to 2007. So if I needed to, if I had a very old script, I could install a binary pre-compiled version of this library from 2007 and install it. That binary component is really important. Inside Linux, packages actually have to be built, but we provide
binaries that are pre-compiled. So this means that rather than it taking like an hour to install a package, it's almost instantaneous because you're just downloading the package and installing it rather than compiling it from source. So that's another benefit that Package Manager provides.
You can also see here on the left, we have different repositories or collections of packages. So this basically means we're able to curate specific aggregations of packages and what they look like. So I can have internal only for packages that we've written inside like RStudio that aren't public or for your organization, packages that you're writing to do very specific things that you don't necessarily want to make fully available and you're just using internally.
And then Package Manager also supports PyPI. So you can make a copy of PyPI and let's see if we can get pandas. And we can get pandas. We can get the latest version of 1.3.3 as well as older versions going back to older releases. So again, both R and Python support.
There's a question about integration with Tableau. Stay posted. There's some hopefully exciting news coming out quickly. And then for team of government consultants able to adopt the whole tool chain internally but able to deploy potentially extensive data dashboard for clients.
So what we see a lot with teams that have multiple constituencies is you can either run multiple servers. So like some groups have an internal RStudio Connect or internal RStudio Workbench. And then they have another version that's actually for external use. So again, you can either have people completely in a completely separate environment. So there's no chance of exposure of data or reports. Or you can add them into your authentication group and just allow them to see very specific components of very specific reports.
In terms of scheduling, that can be done. There's another question about that. So scheduling can be done down to the minute or up to the year. So it doesn't have to be daily. It could be every five or 10 minutes. And you can have it run however frequently you want. As far as package manager, it can support CRAN as well as Bioconductor or GitHub. So yes, you can have a package that only exists on GitHub. You can install it onto RStudio Package Manager. Or for example, if you had an internally developed package that didn't exist anywhere else, you can also install that locally on your RStudio Package Manager without exposing it to the world.
So we're the last few minutes here. I've answered a few questions. So let's dive back into the presentation to wrap it up. And then I'll stick around longer than the top of the hour and continue answering questions as long as folks want to. So let's dive back into this. There's still some more components here that I want to talk about. So we got to the live demo portion. And we're going to talk real briefly about some of these other components. And then I'll answer some more questions.
Customer stories and reviews
So while I'm really excited about RStudio team and all the different amazing things it can do, we often you want to hear from what your peers think about this. So if you go to our customer stories pane, which again is linked here, or you can go to rstudio.com slash about slash customer stories, you can read about what other data science teams have done with our products, basically how they've used RStudio to harness the full capabilities of their data science team, or streamline their deployment process, or merge software engineering and data science without making it painful for everyone to interact with it. So these are actual real life stories of how other people out in the wild and your peers are using our products.
If you'd rather just kind of read summaries or reviews, these are public reviews on trustradius.com. And these are all verified. So this means that they're customers, we're not able to kind of influence them, they're just putting it up here on TrustRadius. People talking about a mix of R and Python on RStudio, or how RStudio products are great for their technical teams and their business team members. Integration of both languages helps them out a lot in terms of being able to be productive.
And a lot of people use RStudio Connect as their like one-stop shop for publishing. So documents, apps, APIs, you have a clean professional interface where you can work with all your different clients and stakeholders internally, along with your authentication and along with all of your kind of professional database products.
RStudio Team as a modular platform
So if we think about RStudio team, it's really a modular platform. So you can always choose one of these if that's all you needed. Some teams only have Workbench, some teams only have Connect, some teams have Connect and Package Manager. If you get them all together, there's actually a bundle discount, but you can always purchase them individually if that was the only data product you needed. But importantly, this is complementing your existing analytic investments. Every organization has different needs. Every organization has all sorts of different data products or kind of integrations that they want to bring to the table.
So your corporate data, all the different databases you're using, things like Spark or distributed computing, basically any database that you can connect to through ODBC and R and Python, you can pull into RStudio Workbench or RStudio Connect. You can also connect to those distributed environments. So someone had mentioned Databricks. So DB Connect is a great example of being able to send compute into giant clusters in Spark and then pull down the results into, say, a Shiny application.
You can also connect to external job management clusters. This is not required in terms of you don't have to be Kubernetes native, or if you don't know what that word means, that's okay. But Kubernetes is basically a way of doing elastic scaling of your environment. So rather than having a traditional server that has a set amount of RAM and memory, Kubernetes allows you to spin up new compute as needed and then shrink back down. And that's supported inside RStudio Workbench, where you can actually send out those large jobs to be executed and then close down the environment so you're not paying for it all the time.
You can connect to other data science tools. So again, because you're using R and Python, and these are open source data science languages, you don't only have to use them inside these products. Just we think they're great environments for that. R and Python can be used in anything else. Maybe you also have a Docker-based workflow, or you have other data science products that use R and Python. We're not changing any of the R and Python open source in terms of we're just contributing open source software, but we're not flavoring it where it only works on here and doesn't work on there. You can take those open source work that you've done and use it anywhere. So you're not locked in or limited to proprietary formats.
You can also connect to things like reporting or BI tools. So a lot of the team, a lot of the time, there's actually a separation between your business intelligence team and your data science team. What we're seeking to do here is make connections or support between those. So maybe you have an automated job on RStudio Connect that updates data that the BI team uses. So rather than your business intelligence analyst coming to data scientists for ad hoc request, they have a self-service or automated tool getting that data, fitting those models, doing some type of work in R and Python that they can then just consume or adapt.
And then you have obviously your tailored decision support. So you can create those web apps in Dash or Shiny or Streamlit or reports in R Markdown that can be accessed by C-level executives or other data science decision makers within your organization. And then lastly, you have full support for these automated decisions or the ability to publish RESTful APIs in R and Python that integrate into any other software.
RStudio's mission and open source commitment
Now, in terms of closing out part of the presentation for today and something that I'm, again, very passionate about is what makes RStudio different as a company? Our mission and kind of our purpose for existing is actually creating free and open source software for data science, scientific research, and technical communication. Basically, we want to make sure that we allow anyone with access to computer to participate freely in a data-centric global economy. We want to enhance the production and consumption of knowledge, and we want to facilitate collaboration and reproducible research and science education industry. So this is why we create so much open source software and why more than 50% of all of our engineering resources go to free and open source software. Things like all the tidyverse or the R packages we release, some of the new support that we're doing for Python in terms of contributing either financially or with code to other Python products. So overall, revenue from our products goes into funding our open source so that we can essentially achieve this mission.
So this is why we create so much open source software and why more than 50% of all of our engineering resources go to free and open source software.
So we're ecstatic if you can, you know, get everything done with our open source software. That's one of our goals is that you can be extremely productive with all of our software. For some organizations that need things like authentication or scaling, those problems are not necessarily like data science problems. We're solving IT or administration and data science problems. So there's this interaction between our professional software funding our open source efforts for the world.
Additionally, we're a public benefit corp. So this was announced at the 2020 RStudio Conf. So if you haven't seen that video, I highly recommend going and looking at Keynote. JJ Allaire, our CEO and founder, basically helped convert RStudio from an incorporated organization to a B corporation or a public benefit corporation. So this basically means that our open source mission is codified or built into our company charter. So any decisions we make must balance the best interest of our community, customers, employees, and the different shareholders. So definitely something I'm very proud of. And if you have the chance, I would highly look at this RStudio.com slash PDC keynote to read more about that.
Resources and next steps
So I'll stick around for some more questions. But if you just want to kind of read some links or do some things, you can read all about serious data science and RStudio team with these links. Again, this presentation is available at colorado.rstudio.com slash RSC slash RStudio dash team press. If you want to know more about what actual RStudio team users think, you can read our reviews at Trust Radius. Those are all the public reviews of our product from customers. If you do want to chat and get specific questions answered, or you wanted to talk about something and you didn't feel comfortable asking in a group setting, you can always book a live meeting with us and we can connect you with one of our experts internally and we can talk about whatever you need to. Again, no strings attached, just let's have a conversation.
And then lastly, maybe you don't even want to have a meeting. You just want to evaluate the product. So you want to be like, okay, this looked cool, but I want to actually try it out myself. You actually go and evaluate RStudio team today in a hosted environment. You can go actually request this and try it out in the cloud. No strings attached, no purchase necessary and try things out.
I have a few more minutes left. I've been trying to answer questions as they come in, but I'm going to try and go back and see if there's a few different questions about different things. Let me go through those. Yes. So a good, really good question, is Connect completely cloud-based or do we have to do on-prem and get our IT group involved? You have the choice. So in terms of we think of going back to here, you can install this on bare metal servers behind your firewall on-premise. Lots of customers do that. That's great. We fully support that. You can also install it in a cloud of your choosing. So AWS, Google Cloud, Azure, IBM, wherever you want to install it. The only requirement here is that you're installing in a Linux environment. So wherever that Linux environment lives is fine, just a Linux environment. So if you want to do on-premise, that's great. If you want to do in the cloud, that's great as well.
The question about scripts being triggered, say on new data available, you can build in logic to your rerunning of the script. So basically you can have it check every 15 minutes or something and be like, is new data available? It's checking on it. And if no, then stop. If yes, then rerun. We are evaluating the idea of true logic-based scheduling built in our Studio Connect. So if you are interested in that, please reach out and help us promote that idea. For now, it's scheduled based on a time base, but you can integrate some of that logic into the code within the script.
So they want to use that scheduling function, but how to check daily results. So one of the things is like as the admin time, or as the admin of the server, what I can look at is I can see all the different things that are scheduled. So this was a good customer request in terms of like, I schedule something and I didn't realize it was every minute. It's like, oh geez, I want to look at this and be like, why is this happening every minute? Sometimes that's actually necessary. And you can take a look at like, okay, this is occurring every 10 minutes. Maybe I should check on it and can check with Ian about if that's supposed to be scheduled as well. Or I can just look at all the different kind of things and when they're scheduled.
And what I can do with this view is actually separate out computationally intensive portions, so like model training, to occur overnight. So say like between midnight and 5 a.m., I do all this computation when no one is using the server. And then at 6 a.m. to 5 p.m., that's when I leave computation available for business users to come in and interact with my applications.
So I know another question was about if there's a link to the GitHub repo. So if you did want to look at some of the examples I showed in terms of these are actual code examples that you can play around with. If you go to my GitHub at jthomasmock slash rstudio dash team dash demo, that has all of the different resources I showed today and has a link to some of the other portions of what we covered today as well as the slides.
I'm going to try and see if there's any other questions. I've got to move around some pages. Tanya, any other questions that came in or any other links you'd like me to share?
Pull this up, go back in the workbench.
No. Okay. I think that's all the questions for today. But if you do have more questions, please do reach out to us. Again, if you go to the last page of this, you can actually schedule a meeting or reach out directly. Or you can reach us at sales at rstudio.com
with general queries. And we can always talk to you by email if you'd rather not have a live meeting. But we're very passionate about talking to customers. But we're very passionate about talking to customers and hearing from y'all or potential folks in terms of what types of things you need to do in your organization to be successful.
So please do reach out if you have any questions. With that, thank you so much. One more question coming in. Does RStudio Connect still require root-level privileges to run? So RStudio Connect, if you're talking about running Connect inside a Docker container, it still requires root-level privileges. But if you want to talk further about that, please do reach out. We have some very exciting kind of work happening in that area. So if you're interested in kind of talking a bit deeper about root-level privileges, please open up in chat to us and we can talk about it.
Let's see if I can pull up. Got to pull up the one page. Thanks so much for kind of learning with me today. We're still kind of evaluating how best to use some of these YouTube live events. So if you do have feedback, please kind of reach out about that. I'm going to look and see if there's any more questions. Appreciate the thank yous. Let's see. Yeah. And then some questions for Tanya. Again, if you want to evaluate, there's the link that was dropped in the chat about the evaluation. Still got 100 and something people in the chat. So if you have any more questions, please keep them coming. Otherwise, I can kind of drop off here in a few minutes. But thank you so much with your questions today. I'm very excited about this kind of stuff. So thanks for kind of learning along with me and enjoying this YouTube live event with us. I see one more question coming in. So I'll give that one a second to roll.
Tableau and Power BI integrations
Yeah. So there's a question. I don't want to steal the thunder. But we've had a couple questions about Tableau. So let's talk a little bit about that. So let's go back to content. So, yes. In short, we do have integrations. There's the Shiny Tableau package, which is a way of embedding Shiny into RStudio Connect as well as interacting with Tableau. And I actually have an example of some content we're working on internally on making those connections. So the Plumber Tableau package for R or the fast API Tableau package for Python actually allow you to directly create RESTful APIs in R or in Python that directly connect with Tableau via the Tableau analytics extensions. So this is kind of a first class citizen of R and Python running within Tableau. And again, trying to merge those ideas of data science teams and business intelligence teams working happily together as opposed to being at odds with not being able to, you know, not being able to interact together and not being able to, like, work together cleanly. So these two packages are very exciting. So basically, full R and Python support for analytics extensions and Tableau. And those can be hosted on RStudio Connect just like the other kind of data products we showed today.
And again, trying to merge those ideas of data science teams and business intelligence teams working happily together as opposed to being at odds with not being able to, you know, not being able to interact together and not being able to, like, work together cleanly.
Question one more. Connect servers fail to support email. Yes. So if you have a question about email, please do reach out to support at RStudio.com. And we have some resources around that.
Another question about Power BI. So Power BI, I believe, actually has a support for RESTful APIs. So without even having to do anything special, just using Plumber in R or fast API or, you know, Flask in Python, you can actually call that from a Power BI notebook. So as long as you can query a API in terms of we think about the apologies for bouncing around a little bit trying to get back to this presentation so I can show a quick example. So bear with me for a second. Thanks for your patience. But I see you asked that twice. I want to make sure I get you a good answer.
Let's look at this. So RStudio TuneDemo. So the short answer is so these APIs, if you think of it like an API is just an endpoint that exists and can be queried from anything that can communicate with it. So my understanding is that things like Power BI or other software can actually interact directly with those APIs and just perform a query. So you could actually have an API written in R or Python living on RStudio Connect. And then you're querying it in things like even Excel. I believe that Excel even has support for some API capabilities. Maybe that's not the ideal workflow, but it is definitely possible.
Starting to lose a few people. So I know we're about four minutes over. So thank you again so, so much for joining me today. It's always a pleasure to kind of connect with the community. Thanks for taking the hour out of your day to hang out with me for a while. Definitely if you want to stick around for other things, our data science hangouts are happening, and those are great for more kind of informal discussions or just talking a bit more about data science and being a data science practitioner. But thank you so much. And if you have further questions, please do reach out to us here at RStudio. Always happy to chat. Other than that, have a great weekend, stay safe, and thank you for your time.