image: thumbnail.jpg
Workflow Demo Live Q&A - May 29th
How to automate your reporting with Quarto Dashboards and Posit Connect
Anonymous questions: pos.it/demo-questions
Demo: 11 am ET [https://youtu.be/xnJuXOw7iu8]
Q&A: ~11:30 am ET
Transcript#
This transcript was generated automatically and may contain errors.
Awesome. Well, let's get started here, and people can join us as they do.
Thank you all so much for joining us today, and thank you, Isabella, for such a great session. I'm going to pull Isabella and Charlotte here on stage with me. There we go.
As a reminder to everybody, we have these Workflow Demos the last Wednesday of every month, and they are all recorded, and so I'm just going to share the link in the chat with you right now. So if you want to add the monthly recurring demo to your calendar, you can use that. But we now have 14 – I can't believe we've been doing it this long now – 14 different workflows shared as recordings, from building a model annotation tool to pins workflows to deploying APIs, and you can see those all here in the playlist on YouTube as well I just shared in the chat.
But also just wanted to mention real quickly, if you are new to Connect or just curious to try it out after Isabella showed you everything that's possible, you can request a free evaluation from our team, and I just put the short link for that on the screen there, and I'll put it in the chat too.
And then before I jump into Q&A, I also wanted to let everybody know, if you're interested in learning more about Quarto, there are three Quarto workshops at PositConf this year in Seattle in August, and so there's one on building a dashboard, on introduction to Quarto, and on Quarto websites. Isabella, did I miss any? Got it. Okay, cool. I'll share the link to the workshops there too.
But also, if you are thinking about joining us at the conference in Seattle in August, I want to also invite you to join us with the Posit team at next Thursday's data science hangout, and so at that hangout, we're going to answer any questions you may have about PositConf, from workshops to talks to advice for first timers, and it's also a great opportunity to meet other people ahead of time in the chat who will be there in person too.
Okay, I'm jumping back into the workflow mode. For the Q&A room here, if you'd like to ask any questions anonymously, you can use the Slido link that I shared earlier, which I'll share again here. It's pos.it slash demo dash questions, or just feel free to type right into the YouTube chat.
And before we jump into the questions, I thought we should go around here and just say hello and introduce ourselves as well. I didn't do that yet. I'm Rachel Dempsey. I lead customer marketing here at Posit. I'm based in the Boston area, calling in from our office today. And I'm also joined by Charlotte, helping with the Q&A, and Isabella. Charlotte, you want to introduce yourself?
Hi, everyone. My name is Charlotte Wickham. I'm on the developer relations team at Posit, and I work really closely with Quarto and the Quarto team.
And Isabella, I know you introduced yourself already in the demo, but why not again?
Hi, everyone. I'm Isabella Velazquez. I'm also on the developer relations team here at Posit. And thank you so much for watching. Looking forward to your questions.
Questions from the audience
So I pulled a few of the questions over from Slido already that we can get started. But while we just talked about conf, I thought this would be a good one to pull up first. Someone said, thanks for presenting, mentioning Posit Conf. It's a good hint to ask again when virtual participation will open.
Awesome. Okay. One question we saw come in during the demo was, what suggestions do you have for developing a Quarto dashboard over a Shiny app?
Great question. I think it is kind of a balance of the simplicity of what you're looking for, what you hope that your audience can see and do, and then also the resources that you have available for actually publishing and sharing your content. I would say there's definitely a spectrum of very, very simple kind of quick prototypes all the way up to very complex, robust, scalable, interactive apps.
Quarto dashboards definitely can hit on that kind of like, if you need something quick and simple, and Shiny apps can become, again, more complex depending on your needs. One nice thing about Quarto dashboards is that you can add interactivity with Shiny. So if you're kind of looking for that middle ground of not doing a full-blown Shiny app, but you also need toggles and filtering and things like that, Quarto dashboards are still an option for you.
Yeah, I would just add that, you know, there's sort of this definite like deliberate consideration on both the Quarto and Shiny team side that it should be easy for you to move, right? If you start in a Quarto dashboard with your prototype, it's relatively simple to then add in some Shiny components. And if that gets to the point where it's just not doing what you need it to be able to do, then you can move up to a full-scale Shiny app.
Yeah, I would just add that, you know, there's sort of this definite like deliberate consideration on both the Quarto and Shiny team side that it should be easy for you to move, right? If you start in a Quarto dashboard with your prototype, it's relatively simple to then add in some Shiny components. And if that gets to the point where it's just not doing what you need it to be able to do, then you can move up to a full-scale Shiny app.
Great. Thank you. I see a ton of questions coming in now on Slido. And one that someone just asked was, is there any way to chain the refresh of the dashboard to the refresh of the Python script? So use the presence of new data as a trigger.
Yeah, I think you could get really creative in terms of how you would like to chain things. Of course, today's demonstration was very simple, like schedule the script, schedule the dashboard. But Posit Connect has various options for kind of like this automatic refresh, for example, Git-backed publishing. So anytime the Posit Connect sees a change in a GitHub repository, it can automatically schedule that refresh of, say, the dashboard and things like that. Of course, there's various other tools and suites of tools for continuous integration. So my answer would be yes. But I think, like, as you want to make it a more complex kind of chain, it really depends on, again, the tools that are available to you and how exactly you want to do it.
I know we were talking about this one a little bit earlier, too. And it was, can you say a bit more about creating a new virtual environment? Is this a Python thing? Would it be relevant to our own development?
Yeah. I would say in my experience working with Python, virtual environments are very, very important and helpful. So they create this kind of isolated package of, like, your packages, your Python installation, and make it a lot easier for whenever you need to reload your project or you're sharing your project across environments, like if you're doing it locally or on Posit Workbench, like you saw in the demo.
So, again, my understanding is there are tools that can do the similar sorts of tasks in R. One example is the R package, which creates kind of a list of the packages that you're using in a particular project, the version. And so that way, if you are working with other people on other projects, you can ensure that everybody has the right packages installed of the right version, things like that. And there are other tools if you need to, say, control the version of R itself and things like that.
I would say in my experience, when I first started using R, I never really worried about that kind of, like, environment recreation. I was working as a solitary data analyst, and I knew exactly what package versions I needed. And as things became more collaborative and more complex, as more people got brought into the team, then it became very important to make sure that everybody was aligned in terms of their packages and their package version.
One other thing to mention about Conf is that there is a workshop called What They Forgot to Teach You About R. And I actually took that workshop in Seattle back in 2018. And that's when I learned about RMV and how to kind of create these isolated environments where collaboration is a lot easier when you need it in R.
Thank you. Charlotte, do you want to add anything to that?
No, I think that was an excellent answer, Isabella.
Awesome. I just put into the chat a blog post that I believe Isabella wrote as well on all of the workshops at the conference. And I was just playing around with the interactive table in the blog post, Isabella. I was just wondering how you did that.
Yeah, that table is made with an R package called GT. And so it makes it very easy to make these beautiful, filterable, sortable tables in R. It took me a while to realize you can create the filterable table with GT. So that's an awesome example to see. Thank you.
Let me see. Oh, one other quick question about pins was, can you pin data to AWS S3 buckets?
You can, yes. And I will try and grab that documentation while I'm reading here too, and I'll put it into the chat.
Let's jump over to see if there were any in YouTube. Oh, some love for the GT package there.
Here is, let me just copy this one over. Bunch of questions in Slido. This one is, can you set different hierarchy of login details? So like head of service sees data set one, manager sees data set two, or employee sees data set three.
Yes, I believe so. You can create different kind of groups for permissions on Connect. So in the demo, for example, you saw anyone, specific users, or just me. And in that specific users, you could kind of create these different groupings depending on the permissions you need.
I think that ties into, there's another similar question that was, when using Quarto dashboards, can I have row level security? Or is it just a static interactive HTML, which all users would see the same information?
Good question. Charlotte, any thoughts?
I'm not super familiar with the term row level security, but my general understanding, no, like the dashboard that you create from a Quarto dashboard is an HTML document. So the level of permissions you're going to be able to put on that sort of more controlled by where you're publishing it. You could imagine potentially publishing different versions of the dashboard to different endpoints on Connect and then controlling the permissions that way.
I just want to just share in the chat here. I know that's something that comes up a bit on. So years ago, I used to be in sales at Posit, and that would come up on a lot of customer conversations with solutions engineers. So I just put a link into the chat if you do want to speak with our team about that and dive deeper into any of the questions on security too. We're happy to chat more about that.
One other one was, sorry, I'm a little slow with copying these over from Slido. It's not the same as when I just show them from YouTube. Okay. One was, can we create reactive Quarto dashboards or do we need to move to Shiny? So, for example, if we want the value boxes to change dynamically.
Yeah, you can create interactive Quarto dashboards. The one in the demo today was just a static one, but you can start embedding pieces of Shiny in for when you want to, say, have a dropdown that filters a plot and things like that. And as mentioned earlier, as your project becomes more and more complex, it's an easy transition to a full Shiny app when you need it.
Comparing dashboarding tools
Okay, I might get one of the pronunciations wrong of this first tool. But how do you compare or choose dashboarding tools among TyPy, Quarto dashboards, Streamlet, and Dash Plotly for production and scalability purpose?
Yeah, I mean, I think it just, again, comes back to what you have in terms of resources. One nice thing about Posit Connect is definitely Quarto dashboard, Streamlet, and Dash. You can publish all three of those on Posit Connect if you need. I also have not heard of TyPy, but I think it, again, just depends on the goals of the project that you want. And I think, yes, sort of like tool chain that you have readily available in terms of the dashboarding tool that you ultimately choose.
Okay, let's see. And Charlotte, or maybe you want to answer this one. What's the advantage of using a PIN first, just loading the data directly?
Actually, my area of expertise, Isabella, if you have an answer, I mean, I can...
Oh, sure. Well, I really like using PINs because, well, it enables to do that sort of refresh, as you saw on Connect. You PIN the PIN, and then that script that creates that PIN can be automatically refreshed. And because it's like the certain file format, it's very small, it's portable, and PIN also allows...
There are a few functionalities that I actually didn't get into during the demo, but one of the things is like PINs has versioning. So say, like, I want to go back to a previous PIN, I can go ahead and call that specific PIN that I need and put it in whatever project that I want. I love how versatile PINs are. So PINs can be data, they can be models, they can be APIs. It's so, so nice in terms of having that flexibility, the things that you want to do.
Yeah, and then with that flexibility, it's like, as you saw, you can pull the data from an API, clean it up, and you never have to save the CSV somewhere. Remember where you put it, remember what version you were using. It's all on Connect. All, you know, labeled, documented version. I use PINs a lot in my projects.
And I guess I'd just add, like, just to make it really clear, like, you could have it in a CSV, right? But when you read in a CSV, there's often a lot of things you have to do to get the data exactly the way you want it. Like, make sure your dates are written as dates. If you've got factors, make sure they've got the right labels or have an order. Like, you don't have to worry about that with PINs. That's one of the advantages.
And then the other thing I just mentioned is, if you did go to Shiny, you could have your Shiny app, as it's sort of set up, read from a PIN. And that way, it's always going to grab, like, the up-to-date data. If you've got that PIN on a schedule, you're going to grab the latest one.
PINs is one of those packages that it feels like whenever we talk about it, there's so much love for it and a lot of questions about it, too. So I just wanted to share in the chat that Ryan Johnson did one of these workflow sessions on all things PINs. And so he walks through six different PINs workflows. And I just put that into the chat. Sorry, I should put descriptions with when I share it. But that's the PINs workflow demo right there.
Quarto book and Python interoperability
This one, I'm not sure if you both know this, but somebody asked, when will there be a high-quality print book, such as those authored by Hadley, which covers this material and the use of Python in these Posit products? Is there a Quarto book? Is there a Quarto book? There is an in-progress Quarto book. Unfortunately, it's not authored by Hadley Wickham, but it's authored by me and Mine Chinca Rundell. So hopefully we meet that bar. And, yeah, keep an eye. Like, it's in progress. We don't have a lot to show you right now, but we'll let you know.
A question just came in about, is there any package in R or library in Python for data validation for data pipelines for dashboards? I'd have to do some research. I don't know any off the top of my head. And, again, these are all great questions everybody's asking. It's kind of hard for us to ask follow-up questions when you're not in the room with us. So if you do want to chat with the team or schedule some follow-up time, there are links in the chat to do that.
Oh, there was one about, could the same be demonstrated with RStudio in R? And I was wondering if there was a great example of that to share. I did want to mention that this could be replicated in R. So all of these tools, one of the great benefits is that they're all interoperable. So Quarto works with Python and R. Pins works with Python and R. In addition, the functionality that you saw in Connect in terms of publishing scripts, scheduling scripts, publishing dashboards, scheduling dashboards, all doable in R. And so, yeah, we wanted to showcase a Python example, but just know, yeah, totally doable.
Using the same PIN across projects
One, I think we briefly touched upon this because you mentioned using the same pin for Shiny apps or Quarto dashboards. But somebody had just asked, can you use the same pin through different Quarto documents? Yeah, absolutely. And again, that's one of the great benefits of pins, too. Say you're working with someone else and you're like, hey, I want you to also be using the latest version of this data all the time. You can share your pin with them. And they like every time that they're running their project, they have the latest version from the pin. And every time you run your project, you're running the latest version from the pin. So you can use it in different places with different colleagues.
What's exciting about Quarto
Awesome. Well, I would love to ask you both. What is something that you're really excited about with Quarto or maybe something that came out in the latest release or just something you're excited about for the future?
So my personal most excited thing about Quarto is the integration with TypeScript. So that's T-Y-P-S-T. That's like a new tool open source for generating PDFs. It's sort of like a LaTeX competitor. And as someone that's always struggled with LaTeX, it's that point in LaTeX where you need to dive into a style file like that. I don't really get it. I hate it. But TypeScript makes it so much easier. The language of types is just so much more familiar to someone like me that's coming from R. I expect the same from someone coming from Python. But it makes it so much easier to customize a PDF. So I think that's going to be really exciting for anyone that has to make PDFs and has to style them in a specific way. I think that integration is going to be a game changer. It's an in-progress integration. So types are still maturing and developing. There are a few things that aren't quite there yet. And the Quarto integration with it is still developing. But I think that's a really exciting place to keep an eye on.
I think that integration is going to be a game changer.
Yeah, I think one thing is just generally the styling in Quarto is so robust. And also there's a lot of new exciting things coming up for styling and branding. And I love just playing around with different variables and tweaking things to make it look exactly the way I like. And so really excited to continue doing that and then also use the new tools available in BSLib or other packages to make the Quarto document of your dreams.
Publishing dashboards and upcoming Q&A
I think we have time for maybe two more questions here. But I did just want to address really quickly one for me was if I could repeat the thing I said about meeting people virtually before PositComp. And so I just shared into the chat, if you're thinking about joining us at PositComp, I invite you to join us for next Thursday's Data Science Hangout. So the team, the program committee, team handling all the logistics, everybody will be there to answer questions. But also as an opportunity to meet other people in the chat who are also going to be there in person. I think that was one of my most favorite things about conf last year was seeing people who were friends from the Data Science Hangout getting to meet in person there.
Somebody asked, can you create and run a dashboard without PositConnect as a static HTML, for example? Yes. So if you go to the Quarto website, Quarto.org, there is a section on publishing and you can see various options there. Yeah. And just to follow it, like if your dashboard doesn't have any shiny components, you can still have sort of widget components like Plotly plots or interactive tables. Those do just produce an HTML file. You can host that wherever you want. PositConnect is just a great way to schedule it.
How did you get code completion in this when you were writing Python code? How did I get code completion? I don't know. I just booted up PositWorks VSCode and started using it, so it might be already integrated in there. Yeah, my suspicion is probably they have the Python extension automatically loaded for you. So if you're using VSCode and you don't have Python code completion, that would be my suggestion to start would be to install the Python extension.
Favorite Quarto examples
Do you have a favorite or a Quarto example that comes to mind that has been most impressive to you? I love getting to see the gallery. I know there's an awesome Quarto GitHub repo as well. Yeah, I love this website. And usually when folks are like, oh, what kind of website can I create with Quarto? I point them to this one. It's World World Data Science. And I just think it's an amazing showcase of just how custom you can make your Quarto website look. And the creator of this website actually gave a talk in the last PositCon. So the video recording is on YouTube to hear about his journey from journalist to programmer.
Yeah, but one thing I've been doing with that. So I'm going to be teaching the Quarto websites workshop at comp. So I've been looking at a lot of examples of what people are doing with Quarto. And I'm still sort of amazed. One of the things that really amazes me is the things people can do with these things called listings. So a listing in a Quarto website is essentially a way to generate a page automatically from some content. And that content could be other Quarto files. It could be just some YAML, some structured data. And people just do really amazing things with these. The sort of classic thing is you create a blog. A blog is just a listing. But people do really cool things to showcase their projects or to list out all the talks they're giving. It's like one of those Quarto features that you read about on the website and you're like, oh yeah, I get it. And then people have run with it and they've done some really, really beautiful, amazing things.
It's like one of those Quarto features that you read about on the website and you're like, oh yeah, I get it. And then people have run with it and they've done some really, really beautiful, amazing things.
I love that too. I won't say that my example is beautiful. But I did use that listings functionality to build a data science community builder site. And so I was pulling together a bunch of resources from other community builders. And I found the listings site to be really helpful for that.
Well, I know we just hit the top of the hour here. So I just want to say thank you so much, Isabella, for an awesome demo. And thank you so much, Charlotte, for also joining us here for the Q&A. I'm really excited to read through everybody's comments and thoughts and see how you all use some of these workflows and see some of the dashboards that you all build as well. But if you have any suggestions for us for these workflow sessions, please feel free to put them into the comments. Or if you ever want to just email me directly, I can put my email there in the chat as well.
I just want to say thank you. Thank you so much. Thank you all for the great questions. Hope to see you again at next month's workflow or at the data science Hangout, or last but not least, at the conference in August. Have a great rest of the day.