Resources
Mar 22, 2024 |
1h 8min
|
5.1k views
|

R/Medicine: Quarto for Reproducible Medical Manuscripts

In this webinar, Mine Cetinkaya-Rundel, Professor of the Practice of Statistical Science at Duke University,  presents a new capability in Quarto that provides a straightforward and user-friendly approach to creating reproducible manuscripts that are publication-ready for submission to science journals. 

This new feature, Quarto manuscripts, includes the ability to produce a bundled output containing a standardized journal format, source documents, source computations, referenced resources, and execution information into a single bundle that be ingested into journal review and production processes. 

In this talk, Mine demonstrates how Quarto manuscripts work and how you can incorporate them into your current manuscript development process. She also touches on pain points in your current workflow that Quarto manuscripts help alleviate.

Main Sections

00:00 Introductions
03:05 Full complexity spectrum of reproducible of reproducible scientific projects
08:14 Leveraging Quarto for fully reproducible scientific manuscripts
16:38 A new project type: manuscript
17:12 Let's write a manuscript
40:41 What's next?
44:39 Thank you/Q&A

More resources

R Validation Hub Site: https://www.pharmar.org/
Main Site: https://www.r-consortium.org/
News: https://www.r-consortium.org/news
Blog: https://www.r-consortium.org/news/blog
Join: https://www.r-consortium.org/about/join
Twitter: https://twitter.com/rconsortium?lang=en
LinkedIn: https://www.linkedin.com/company/r-consortium/
Mastodon: https://fosstodon.org/@RConsortium

image: thumbnail.jpg

Transcript#

This transcript was generated automatically and may contain errors.

Good morning, good afternoon, or good evening, depending on where you are in the world. And welcome to our webinar, Quarto for Reproducible Medical Manuscripts, brought to you by the Art Consortium.

The Art Consortium works with and supports key organizations developing our software through grants and sponsorships worldwide. So please visit our website to learn all the details and how your organization can become a member.

My name is Elena Quintero and I'm today's announcers. And I just have a few housekeeping items before we begin the webinar today.

So basically this webinar has an interactive Q&A section between you and our presenter. So just type in a question into the question windows at any point in time during the presentation and make sure to click the submit button. Near the end of the webinar, we'll try to answer as many questions as we can.

Okay, so let's get started. This webinar focuses again on Quarto for Reproducible Medical Manuscripts. And we have Mine Shintikaya-Randall here with us today, who is a professor of the Practice of Statistical Science and the Director of Undergraduate Studies in the Department of Statistical Science. She's also an affiliated faculty member in the Computational Media Arts and Cultures Program at Duke University.

Her work is dedicated to advancing innovation in statistics and data science pedagogy and focusing particularly on computing, reproducible research, student-centered learning and open source education. Mine, thank you so much for being here today. You can go ahead and begin with your presentation.

Wonderful, thank you for having me. Let me go ahead and share my screen. Is that looking good?

Yes, perfect. Wonderful. Yeah, thanks for having me. It's lovely to be here and I'll talk a little bit about Quarto.

If you have seen me speak about Quarto before, you probably know that it is a topic I'm quite enthusiastic about. And as a manuscript is something that sort of bridges the gap between my two lives, my role at Duke and also my developer educator role at Posit. So I'm excited to sort of talk to you about this.

And while the main talk is going to be about Quarto manuscripts sort of at a high level, I'll also present one sample manuscript that is more of a medical manuscript. There isn't a whole lot about Quarto manuscripts that's very specific to sort of medical cases. It just happens to be a really great tool for publications. And so hopefully the domain specific example will help sort of see that link a little bit better for the participants.

The complexity spectrum of reproducible scientific projects

So let's talk a little bit about sort of the full complexity spectrum of reproducible scientific projects. At its simplest level, you might have a single Quarto document where maybe you're just using art for computational cells in there, and you can run all of your code in a single file and you don't mind running it over and over again with each edit.

This is the sort of thing where maybe this is your homework assignment for a data science 101 course, or even a final project for a stat 101 course, a blog post where you're walking through some data analysis example, but there isn't something super computationally heavy going on, or a tutorial, or even a not too extensive consulting report. Maybe there's just one or a few data sources that come in from a static rectangular file.

You bring them together, you maybe do some modeling on it, but again, nothing too computationally expensive. And in that case, you might put sort of your narrative and your code in a Quarto document with computational cells in it, render it to HTML or PDF or something like that, and you may not need anything more than that.

Something a little bit complicated, but still simple, is one where you can run all of the code in a single file, and you don't really mind running it over and over again with each edit, but you need an output that conforms to a particular journal style. And for that, the journal articles extensions for Quarto are super helpful. There's a whole repository of them. If this comes up in the Q&A, I'm happy to share links for that, but there's a growing number of journals for which there are Quarto extensions, and these extensions sort of allow you to format your document from a simple QMD file to a PDF or a Word that is formatted with a journal style.

And this would be something like an article that uses computation for data analysis, but something that's not too heavily computational, maybe again, something that doesn't take weeks to run.

But we all know that science is rarely simple. You might have multiple collaborators. Each might have their favorite computing language and a code editor. You might have multiple stages of a project, each with their own level of feasibility of what can be rerun with each edit and what needs to be cached.

So I can imagine something like a data analysis where at the exploratory stage, it's totally fine to sort of run your code over and over as you're an iterate easily when you're exploring the data, making some simple visualizations for your own use. But when you get to a point where maybe you're building some models that take a little bit of time to run, your needs might change.

So something more complex could be one where you have a Quarto file, where you have your narrative and maybe some code cells in there as well. But you also have a bunch of R scripts that lay around in your project. I tend to do this very often where I'll have a sort of a manuscript project where maybe I have something like a data folder and inside of that, a raw data and a process data folder. And I just use some simple R scripts to grab the data, clean it up, reshape it and get it to the point where I'm going to be doing modeling or analysis, write it out and then ingest it into my Quarto file.

And then I might be outputting to something like a PDF or a Word document. Quarto works nicely for this, but you as the individual need to sort of like make sure that any changes you make in these R scripts, especially if they have outputs that are then being used in the subsequent stage, that you end up being the person who is taking care of when computation is run and how often.

Even more complex could be one where you have multiple Quarto files. Maybe these are sort of like notebooks that you're using for some narrative and some sort of intermediary figures as well as some final figures. A bunch of R scripts that some of them maybe R and some of them maybe Python. This might be an individual project where if you're sort of fluent and comfortable in the two languages and you choose to do certain types of tasks in one and another type of task in another.

Alternatively, if you have a team of folks working on the project, some of them might be Python users and some of them might be R users and chances are they're going to want to stick with their own languages. And again, we're sort of like you're having to write to a PDF or a Word file, something a little bit that requires a little bit more finesse than just writing out to HTML.

So as this layer of complexity goes up, how can we leverage Quarto for creating fully reproducible scientific manuscripts?

What is a notebook?

Throughout this talk, I'm going to refer to a notion called notebook. So what do I mean by a notebook? What I mean is that a notebook is a document that contains both code and narrative.

So this could be a Jupyter notebook. It could also be a Quarto document. And I often tend to refer to QMD files as Quarto documents. If you're a long time R Markdown user, you may not necessarily have called all of your R Markdown files notebooks, but let's go ahead and call them notebooks collectively, regardless of whether it's a Jupyter notebook or a plain text Quarto document, because ultimately these are both things that allow you to have sort of code and narrative in the same document and execute the code in that document.

The current state of scientific publishing

So what is the current state of affairs? Most computational science, honestly, is born in notebooks. You start sort of fussing around with your data at the beginning of your analysis, and then it builds into a bigger project. And it, well, either dies or ends in Word documents. That ends up being the culminating thing that you submit to a journal.

And for many projects, that's where things end. That ends up being the published work. And oftentimes, by the time the authors of that project get to that point, they are done with it. They probably don't want to see it for a little bit of time. But you, as the person who might be reading that paper, that's exactly when your journey with that data starts. That's when you want to be able to see the code, interact with it, maybe play around with the raw data. Maybe you find a project that's very similar to something you're working on, and you are ready to get into the nitty gritty of it.

But at that point, the folks who have originally worked on this manuscript have packaged things up in a PDF or a Word document, and it doesn't tend to be interactive anymore.

But at that point, the folks who have originally worked on this manuscript have packaged things up in a PDF or a Word document, and it doesn't tend to be interactive anymore.

This notion of, I mean, the peer review and publication workflow, in a way, do not support notebooks as research outputs. They ask for the static Word or PDF documents. Additionally, more complex scenarios involve a lot of manual finagling to bring the project to a journal submission stage. You might have a bunch of script files in a variety of languages, maybe a bunch of notebooks, where you bring things together.

But ultimately, if you don't have the style file that goes along with that journal that you might be submitting to, you're probably going to be sort of breaking the reproducibility cycle and maybe manually editing that document. So often during this process, reproducibility is lost, or even if it's not lost, it can take a second seat to the formatting requirements.

And ultimately, final submission rarely captures all computations, which are at best relegated to supplementary materials. We may have all seen papers that might say, sort of in the appendix, say code is available in the appendix, and then you go to the appendix, and it says code can be found in this GitHub repository.

And then you go to the GitHub repository, and you can tell that this is just code that just got sort of like put in there at the very end of the authoring process, not necessarily code that grew with the project. And I should also mention that probably your final submission doesn't need all of your computations. It's not meant to be a diary of everything that you've done along the way. It really is meant to capture sort of the highlights of the research process, but it would be really nice to be able to sort of capture everything that happened along the way.

And not just the selected figures and tables. So if what I'm saying resonates with you, know that you are not alone in thinking that there is some work that we should be doing in terms of sort of bringing the publication process up to the high standards that we hold ourselves to in terms of doing reproducible science.

Notebooks Now and the roadmap for reproducible manuscripts

And what I mean by you're not alone is, well, I certainly agree with you, the developers of Quarto certainly agree with you, but also there are many other folks, there are many other initiatives. One of them is called Notebooks Now. I've linked to their paper here and I'll share the link to my slides at the very end as well. If you're interested in the work that they have done over the last few years, since 2019 or so, and it basically brought together sort of folks from the development teams of Jupyter Notebooks and R Markdown and also Quarto as well along the way.

And the idea was to sort of start creating these fully reproducible manuscripts that also embed the computation along with it. And what I'm going to be talking today is basically an effort sort of in that direction as well.

So roadmap to fully reproducible scientific manuscripts that are not just PDFs, that are the outputs of a single QMD file, and which I think this represents a lot of scientific work that's being published today. So we need an end-to-end scholarly publishing workflow that treats Jupyter and Quarto notebooks as primary element of scientific record. Another piece of the puzzle is that we need a publication process that elevates transparent and reproducible work by authors, where the data and software together with the narrative are documented, shared, and also archived along with the paper.

So the paper that gets published is not just the single PDF, but an archive of all the compute that goes along with it. And ideally, it would be great if there was new forms of credit to the wider research community, including the research software engineers or research, yeah, including research software engineers. So what I'm, and that bit is probably more advocacy from folks who are involved with sort of journal publications, journal editors, or even faculty and researchers. And that's not necessarily what I'm talking about today is going to be addressing, but I do wanna note that as we bring in new tools

and technologies to the sort of the toolkit of researchers, we are asking them to do maybe a little bit of extra work, at least they're having to learn a new tool. And it would be great if there was sort of credit given for this as well.

But what I'm going to talk mostly about today are addressing the first two pieces of the puzzle. So with Quarto, if you're familiar with Quarto, you probably know already that Quarto can be authored in your favorite code editor. It can render from a QMD file, which is a plain text file or a Jupyter notebook, to PDF, Word, HTML, what have you.

You can execute code in R, Python, and more. You can apply journal styles to your outputs with Quarto extensions. I saw that one of the attendees shared in the chat linked to the Quarto journals repository. And you can publish to GitHub Pages, Netlify, and more.

Quarto manuscript project type

Now, you can also, with Quarto projects, orchestrate multiple inputs and outputs. And I will say that with a new project type, starting with Quarto 1.4, which was released earlier this year, you can orchestrate multiple inputs and outputs with embedded computing using this project type called manuscript. And that's what we're going to be looking at today.

So a new project type, manuscript. In addition to doing everything you can with Quarto, you can now produce manuscripts in multiple formats, so including LaTeX and MS Word formats that may be required by journals, and give your readers easy access to all of the formats through our website. And you can also publish computations from one or more notebooks alongside the manuscript, which allows your readers to basically dive into your code and view it or even interact with it in a virtual environment.

And you can also publish computations from one or more notebooks alongside the manuscript, which allows your readers to basically dive into your code and view it or even interact with it in a virtual environment.

Creating a manuscript project

So let's go ahead and write a manuscript. One of the approaches is you can use the Quarto command, Quarto create project manuscript, and then give the name of your manuscript, and then easy peasy, just add your manuscript content, which is all of your science. Alternatively, in the Quarto documentation, you'll see that there's a sample repository that you can clone and start with that.

So sometimes starting with sort of an empty project can be a bit more daunting than starting with something that has some of the pieces that are already working. So if you're interested in that, you can take that too. And I would strongly recommend that you track your project with Git and host it on GitHub for easy publishing. And that's the workflow that I'm going to demo today.

So we're going to take approach one, which is creating a new project. And so I'm creating a project called Indo RCT. And in that project, when I run Quarto create project manuscript Indo RCT, I can see that Quarto gives me some options. It says that it created a underscore Quarto YAML file. What that means is that's the file that has the metadata for how your project should come together. We're going to look into the contents of that in a second.

It also created an index.qmd file. And that file is basically where the content of your manuscript goes. And it also comes with a references.bib file as well, acknowledging that you're writing a scientific manuscript. Obviously, you're going to have a bibliography that goes along with it.

And then in an interactive menu, it asks you, do you want to open it with VS Code? Or do you want to open it with RStudio? Or do you not want to open it at all? You can use Quarto with really any text editor and then run your Quarto commands in the terminal alongside it. But since most of us tend to use Quarto with an IDE, these are the two options given to you. So I'm going to choose RStudio.

And here is what my project would look like at the beginning of my, soon as I launch, sort of create this manuscript. I can see in my files tab that I have that underscore Quarto.yaml file and my index.qmd file. And since I chose to open this in RStudio, I also get an Rproj file that goes along with it as well. I recommend having one of those files in each one of your projects.

And if you sort of like launch a new project from RStudio, you'll get that for free. It allows you to sort of like control some of your authoring experience. And I'll give a couple of examples of options that are available to you in your Rproj file that might be helpful, especially for manuscripts. And finally, your references file.

The _quarto.yaml configuration file

So let's take a look. In that Quarto underscore Quarto.yaml file, this is sort of what you get as a bare bones. We have, we're saying that we're creating a project and the type is manuscript, and our article is in a file called index.qmd to begin with. You can also see that I have multiple formats that I'm creating, HTML, Docx, so a Word document, and a Jax. So that's a bundle of all of your documents that some of the journals accept, or at least the sort of the notebooks now.

Folks have been sort of trying to move journals towards sort of grabbing this archive that goes along with your manuscript to sort of have a full archive of all of the work that has gone into that project. And we can do other formats as well. And we're gonna reveal the PDF format in a little bit when we open up our sample project.

Demo: the finished manuscript

And here's what our finished document looks like. So let me go ahead and make my script a little bit bigger. And I wanna thank Peter Higgins who shared some sample code with me and sort of pointed me to this paper for which there is some, there's a R package with the data available. So I've gone ahead and recreated the paper, not all of it, but majority of it.

So let's go ahead and sort of go forward. And I've also given a, as you can see a disclaimer here saying this is not original work. This is just a reproduction of this paper for the purposes of demoing Quarto manuscripts.

So in this published paper that's currently published this version on GitHub pages, you can see that I have some metadata up here. We'll get to the other formats in a second. And as I scroll down my abstract, I also have some links that would take me to the GitHub repository for this, as well as a binder link. We'll get to that in a second.

And then I have a table of contents that I get by default, as well as some embedded notebooks that we're going to talk about in a second as well. But I'm gonna sort of scroll down to see like what a Quarto manuscript looks like sort of at a high level when published as an HTML document.

Some of the neat things you get are each of your references are hoverable. So you can actually sort of like hover over them. And then when we get to something like figures, let's scroll down a little bit more. And here, for example, we have a reference to our figure one. You can see, I can similarly hover over that and get a sneak preview of my figure or I can scroll down and see my figure that has been created with some R code.

And you can also see that the code for this figure actually lives in another notebook that has been embedded with my manuscript. So we'll demo that in a second as well. You'll be able to see here that I have some numbers that are sort of in my text. We'll take a look at the source code for this manuscript in a second, but leveraging standard Quarto functionality of inline code, here, these numbers are actually not hard-coded, but generated with inline code.

You'll also see that as I sort of like highlight some text, I'm getting this option to either actually do a highlight, let me not log in right now, or annotate. So I have enabled hypothesis, which is sort of an online annotation tool for this project. It takes just one line of code in your YAML file that we're going to see in a second. And it allows you to sort of add annotations, which might be helpful for collaboration.

Addressing the Word round-trip question

I see that there's one question in the chat, and I feel like this is a good moment to address that. We can go from Quarto to Word, but can we go back from Word to Quarto? No, there isn't a straightforward way. That is a Quarto, native Quarto tool for going back from Word to Quarto. So if you use Word for track changes and review of work, you don't have a way of sort of slurping those back into your index.qmd file.

However, if you sort of host your manuscript during the development stage on something like GitHub Pages, your collaborators could also leave comments using this tool as well. So this may be one way of potentially sort of veering away from comments in a Word document, which might be particularly helpful for collaborators who may not be the ones who want to dive into the GitHub repository and write the code.

All right, so I'm going to scroll down to see, we have some figures, we have some tables, so on and so forth. And at the end, we have a list of references as well.

Multiple output formats from a single source

All right, so what are some of the highlights from the manuscripts? We can see that we have multiple formats from a single source. So all of our narrative is in that index.qmd file, but by stating multiple out formats in our Quarto YAML, I'm able to get a Word, a PDF, and a Mecha bundle as well.

So let's go back up here and see if I can actually click on this MS Word file. You can see that I downloaded it. I'm going to go ahead and share my full screen now. So this is the Word file that I was able to sort of create. And you can see the contents are basically the same, even though it looks a lot different. And similarly, if we take a look at the PDF file here, this is formatted particularly for PLoS. And so you can see that PDF file here as well, and we can download the Mecha bundle too.

All right, go ahead and do this. So in addition, in order to get to this, this is what our Quarto YAML file looked like. I have, as you can see, four outputs that are listed. The HTML is what we were looking at, and the other three, Docx, Jets, and the PDF version are the ones that show up as the additional links.

You can see that I have this comment hypothesis true for my HTML file. That's how I was able to turn on that comment functionality. I haven't given additional preferences or customizations for the Word or the Jets output. I could, but in this case, I haven't. And for the PDFs, this is styled specifically to using the journal extension for PLoS.

And you can also see that I have an option to keep the tech file, the intermediary tech file, which I find especially helpful as I work on a manuscript. Every once in a while, there are some tech errors that I'm only able to resolve by simply opening the tech document itself and rendering that and trying to look at the log of it. So it's helpful to have that intermediary file available. It also is one other way you can share with your collaborators, again, who may not be the people writing code, but who might be familiar with LaTeX.

Rich front matter and author metadata

Additionally, in the index.qmd file, in order to enable this sort of multi-format functionality, we have a really rich front matter. So some things are obvious, your title, your subtitle. You can see that along with my authors, at least for one of the authors, I grabbed some additional information as well. I can add things like affiliation and orchid ID and email. I can note if somebody is the corresponding author. And my references are linked here from my front matter. And then I have a somewhat lengthy abstract that goes along with this paper as well.

In the PDF document, from all of this front matter, we only sort of expose the relevant and the required metadata that's required by the journal. And similarly for the Word document as well, it's only the relevant and required metadata get exposed. So you don't have to change the document YAML as you are changing perhaps the venue of your paper, for example.

I also mentioned embedded computation. So let's take a look at that. So what do we mean by that? I'm going to go down to one of my figures.

Let's go ahead. And here we go. So this is one of our figures. It has a caption and it also underneath the caption says, there's a link to a source notebook that says, this is the notebook called Enrollments and Outcomes. You can see that the same notebook is also linked sort of on the side as well.

So you can customize whether you want the linking only under the figure, only sort of at the document level or none if you don't want to do that. And if we go ahead and click on this, I actually get to see the notebook where this computation was done. And I can have a little bit more narrative here in a more realistic setting, maybe even other computation that is not part of the, that is not necessarily part of the final paper.

And ultimately the figure is produced here and simply embedded with an embed shortcode into my Quarto document.

Authoring manuscripts in RStudio

All right. So let's go ahead and take a tour of how to author one of these documents with RStudio. I am going to open this document on my, in RStudio. And my goal is not to give sort of a full account of doing everything about Quarto in RStudio, but there may be a few handy tools that you're used to that might be nice to see that they work for a manuscript or there may be some handy tools that you haven't necessarily explored in Quarto.

Let's start with the embedding computations one since that's sort of the new kid in town. So I'm going to go down to where I had my results here.

So let's take a look. This is one of the other figures that we have embedded. So you can see that I am using an embed shortcode. And in this embed shortcode, I'm pointing to a folder called notebooks. So we can see that here in my files pane. And then inside of it, I'm pointing to a QMD file called incidences.qmd. In this particular example, I've kept only to R code since this was an R medicine talk, but you can have IPython notebooks here as well. And either in QMD documents or IPython notebook documents, you can have either R or Python code.

And that's sort of the neat thing of it while in a single QMD document, you don't get to have multiple languages without the use of something like reticulate in separate QMD documents, this is possible to do. And so you can imagine your figure one being generated, say with Python and an IPython notebook, your figure two being generated with R in a QMD, and then you can embed each one of them in a single index.qmd file.

So if we go ahead and dive into this incidences.qmd file, you can see that this is where I'm doing the heavy lifting of the data analysis. So I'm loading my packages, I am writing my code for creating these plots. We can go ahead and sort of show what these plots look like.

And when I do, I also give a label to my code chunk. And that label must start with FIG, so fig dash. If I want to do cross-referencing and automatic counting of my figures in the Quarto document, and then I can name it whatever I want after that. And I also need to give it a caption as well. And I'm going to, the code cell option for captions is fig dash cat.

So if we just take a note of this label, fig dash incidences adverse events, and go back to our index.qmd file, we'll be able to see that that's how we refer to that particular code chunk and say, include that figure for me and nothing else from that document. And that linking is what allows us to sort of link to that source document in our published manuscript.

You can also see though that not everything has to happen in a separate notebook. So oftentimes for me, I might have some figures that are perhaps more computationally intensive to produce because maybe they're plotting some modeling results or something like that. But I might also have some stuff that's like pretty cheap to calculate that I don't need to create a whole other notebook for. For example, in this case, I have one table, sort of like a table one sort of thing.

Let's go ahead and generate that table. So something like this, like a exploratory table that gives you at a glance, some information about your data that table is just created in my index.qmd file, not in a separate notebook. So I can sort of mix and match based on my needs, how I want certain figures and tables to be generated.

And additionally, as you can see in this document, I have some inline R code that I am able to run and I have code chunks where I calculate these numbers like percentage with a particular incidence or number of patients in the placebo and the treatment group. So on and so forth. And then I'm pulling those numbers in using inline code.

Cross-referencing and citations

So that's sort of how the computational embedding works. Another neat thing is, so let's go ahead and render this again, the cross-referencing. So with Quarto, you have a few options for cross-referencing for how you actually want the cross-references to happen. But something that is constant is that if you are cross-referencing a figure, they must start with fig dash. And if you are cross-referencing a table, they must start with, the labels must start with table dash.

So if I wanted to make another cross-reference to this figure, say at the end of my paragraph, for some reason, one of the things I can do in the visual editor in RStudio is bring up my insert anything tool so that is command forward shift and say cross-reference. And then I can choose from the figures that are available to me or from the tables that are available to me.

Let's go ahead and do this. And then I can render the document, for example, and we'll be able to see a cross-reference to that. If these are figures that are in the embedded notebooks, they're not going to show up in your selection editor. So for something like this, I would need to actually take a note of the code chunk label from the notebook that I am embedding.

And one more thing I wanted to demo was citation. So let's go ahead and take a look at, let's imagine that I actually wanted to cite this paper, which is the actual paper that I am reconstructing here. So I have the DOI of this paper. I'm going to go ahead and copy that. And let's say that I actually want to cite that paper right in this call out box as well. I'm going to once again, well, I can use my insert anything tool again, command forward slash, or I could go to my insert menu and say, I want to insert a cross-reference.

Oh, I'm sorry, not a cross-reference. Say I want to insert a citation and let's go to from DOI, and I can go ahead and paste that DOI. And here RStudio is basically querying an API in order to sort of look for this paper with this DOI. And then I can go ahead, it creates a sort of a label for this reference for me. I can choose if I want that to be an in-text, so without the parentheses or not in-text citation, and I can go ahead and say, insert.

So what happened is I was able to sort of create this bib entry without having to do the copy pasting from Google Scholar that I tend to do. And if we go ahead and open up our references.bib file, and look for this tag, you'll be able to see that that was inserted to the end of our references file. So that just got inserted now. And let's go ahead and render this document.

And our citation has basically been added here.

Let's go ahead and open up that citation tool one more time. You'll be able to see that if you already have an existing bib file, you can easily select from it and it displays the titles of the papers, which makes it really easy to use. You can use a link as Zotero library, if so, if you commonly use that. And in addition to from DOI, you can link from Crossref, Datasite, PubMed, or if you're looking for a citation for an R package, you can go ahead and look for that here and insert that citation as well.

Interactive computing with Binder

So what's next from here? Well, we can actually dive into the code as well. So what do I mean by that? You've seen that you can peruse the code underlying the figures and tables in the manuscript, but what if you wanted to actually interact with the code in a computational environment that's just a click away and that has all the software and packages needed to reproduce the manuscript?

Back in 2019, Nature actually published a paper that basically, or an article that said, there's a paper that was published in eLife that lets scientists play with each other's results. So this basically had these, the paper had these figures. If you clicked on them, it took you to a computational environment and you can basically see the code and play around with things and said things like, what if I change my threshold? How would my results change or something like that?

As of today, those links are dead, unfortunately. So it has been some time, since 2019, but at the same time, we want to make sure that, if we are saying that we have embedded computing, that it actually stays embedded because in a way, it's been a while since 2019, but in a way, you should not expect scientific results to go away in five years and not be accessible in the original format that they were published.

but in a way, you should not expect scientific results to go away in five years and not be accessible in the original format that they were published.

So with Quarto, one of your options is to use the service called Binder. So you can simply say in your terminal, Quarto use Binder in the project where your manuscript is. And remember, I mentioned that we can go to the GitHub repository for our manuscript, but we can also launch a Binder instance. So let's go ahead and click on that.

And it will take a second for this to launch. I have a version of this that I have already launched. So I'm going to go ahead and share that with you because as I said, it does take a little bit of time for the virtual environment to be built. But once it's done building, this is where it lands you. And then you can actually say, suppose you want to actually do your editing in RStudio, it will bring you to a window like this. There's a little bit of messages that are happening here, but this is not my local copy of this project. You can see that this actually is a version of this document that is on my Binder and all of my computations, my notebooks that I have used as part of my manuscript, as well as the manuscript itself are available to me here.

All right, so I had screenshots of those just in case the service went down. Live demos are always a little bit tricky, but we've been able to see that. All of the files are available. You can launch an instance in your favorite editor and then get to editing it.

Getting started and closing remarks

So if you would like to get started working with Quarto Manuscripts, quarto.org.docs.manuscripts is where I would recommend you find and get started again. It will walk you through a step-by-step tutorial with a sample repository.

And before I wrap up, I also wanna put a little plug in for the R-Medicine Conference that will be taking place June 10 to 14. It'll be a virtual conference. And at the link here on the R Consortium website, you can go ahead. Those abstract submissions are open. But thank you so much for listening. I will put these links in the chat as I am taking some notes in case folks wanted to take a look at them. The top two links are the slides that I went through, the hosted version of them on Quarto Pub, as well as the GitHub repository where the slides, the Quarto file for the slides live.

in case there was something in my slides that's Quarto related, not necessarily manuscript related you'd like to look at. But the manuscript that is a reproduction of the medical paper is also linked.

And under the manuscript links, you can see the hosted version. You can go ahead and launch your own binder instance and play around with it as well, or just clone the repository. And if you like, use that as your starting point instead of the version that is on the Quarto documentation. From a Quarto manuscript functionality perspective, they're very similar. The difference is this is sort of in a medical context and that was in more of a geophysical context, I think.

Thank you so much for listening. And I'd be happy to answer some questions in the few minutes we have left.

Q&A session

All right, so let me go ahead and I'll go ahead and start answering a few questions from the chat. One of the questions said, is embed the same as a source function that will execute an external QMD file? So very good question. Not 100% the same.

So what the source function would do is it would like source it, which means that the computation would have to be run again. That is not a requirement in this case. Something I haven't said much about in this presentation, but I can touch on briefly here is that if we take a look at our Quarto YAML, you can see that I'm also using the freeze functionality from Quarto, which basically means that unless I have explicitly touched a QMD file, do not rerun the code in it.

I can set freeze to false, which would mean always rerun the code, or I can set freeze to true, which means never rerun the code unless I explicitly run it. My preference is usually to keep it at auto so that I don't have to be the one remembering, oh, I touched this file, I should rerun my computations again. But what this does is it sort of captures the computational results at the time and stores them in a freeze file.

And basically the figures that are embedded are actually pulled from these. But the neat thing is that it's also not the same as include graphics either, right? So we're not just including the output, we're keeping that link to the code cell where the figure was actually created for your readers or yourself to be able to track your work back without having to include all of that code in your manuscript document.

Another question, what about the data? Some data are clinical and protected, what are your thoughts on this? So this particular approach, creating manuscripts with Quarto does not necessarily address the challenges around working with private data. It's not meant to necessarily address it.

However, my guess is either all of this is happening in an environment where like you're using all of the functionality of Quarto manuscripts, but perhaps in your gitignore file, you are adding an ignore statement for not committing your data files to your GitHub repository. So in a way, the work that you share isn't 100% reproducible by somebody else, but enough of the code is there, the full sort of history of your code is there so that they are able to follow in your footsteps.

And if you additionally include maybe some readme files that say our data was in this format, these were the headers or something like that, that might go a long way. Alternatively, you might be querying this data from a protected environment so that it never actually makes it into a GitHub repository and others who don't have the right sort of privileges to access that data server never get to touch the data, but again, you get to use everything that Quarto and Quarto manuscripts offers otherwise.

Another question in the Q&A tab is, what is the minimum version of R and Quarto and R tools required to produce manuscripts? So the minimum version of Quarto required is 1.4. Currently the release version is like 1.4.5 or something like that. But if you download the release version of Quarto from the Quarto website, you'll be able to see that you can use Quarto manuscripts.

The minimum R version, I don't know off the top of my head to be perfectly honest. However, something recent, but not necessarily the latest one is what I would say. In fact, I don't think that for many things, Quarto does not require a particular R version. If you're working in RStudio, whatever the minimum version RStudio will work with, and it goes pretty far back, I believe should be good for manuscripts as well.

So will the binder instance contain the same versions of R and packages, et cetera? The answer is yes, it will, because it is created with that Quarto use binder function, which actually looks at the environment that you're working in.

And you can see that when you, let's go here, when you actually run that command, it creates a few files for you that you are asked to, here we go, that you're asked to commit and push to your repository. You can see, for example, which version of Quarto being used, so on and so forth. And these are the instructions that my binder then uses in order to be able to, I think this was another one of those as well. So you can see the R version there in order to be able to sort of build that virtual environment.

Let's see. So another question says, my manuscripts often involve more than 10 authors, each of which will edit the manuscript. Any way to incorporate revisions in such a workflow? Well, if all 10 of these authors are familiar with version control, I think this is a solved problem. They can all have a fork of the repository you're working in, make pull requests or push domain directly, but be careful about what they're doing. And that's sort of done.

If some of the folks do not want to be sort of cloning a data, cloning a GitHub repository and editing directly in a Quarto document, in that case, I guess you or the folks in that group who are a little bit more sort of savvy around writing, around authoring computational notebooks, perhaps can provide some ways for them to provide feedback. It could be using hypothesis, as I said, or it could be using just the word output, but ultimately you'll have to manually sort of incorporate their edits into this.

I will say that I'm very sympathetic to scientists not necessarily wanting to learn all of a sudden a whole new language as well in order to be able to doing edits. But if you're able to get your collaborators to the point where, if they're not going to be the ones writing the code, to the point where they might have RStudio installed, perhaps, just that, or maybe a server version of that, linked to the Git repository where they can simply at least pull the changes, the latest changes, and then make edits, especially if all they're editing is the text.

I think the visual editor is a great way of being able to do that without having to learn a whole new computational language and commit and push their results. That might be sort of meeting you halfway in a way where you get some mileage out of using this tooling and they also learn a little bit of something new along the way.

I think the visual editor is a great way of being able to do that without having to learn a whole new computational language and commit and push their results. That might be sort of meeting you halfway in a way where you get some mileage out of using this tooling and they also learn a little bit of something new along the way.

All right, I think that might be answering the next question as well. It says that we can't copy the... I will take a look at how else I can share the links if copying is not possible.

Binder is not a paid service. That being said, I think the fact that it's not a paid service might also mean that it's a little bit unclear how reliable it's going to be sort of further into the future. But other things you can use if you have a GitHub repository that goes with your Quarto manuscript, you can also use dev containers as well. So that's another way of providing a virtual environment. And that's something you can simply do using the GitHub UI as well.

What about typist documents? So currently by default, this is using LaTeX but as sort of the typist ecosystem becomes more mature and the journal article sort of extensions also start to include typist solutions as well. Those will natively be sort of like available for Quarto manuscripts too. Right now, because I'm using a particular journal style for generating my PDF, and that style is provided with some LaTeX style like textile sheets, I'm going through LaTeX, but sort of any sort of development on the typist front should be able to be sort of used by Quarto manuscripts as soon as the extensions become available for generating those documents with the journal style.

All right, let's see.

Another question is, if I have a medical writer, can the medical writer write the paper template and I fill in the values as a biostatistician? Do you think it would bring efficiency and traceability for manuscript production or any other comments? I think this should be quite feasible. I think we go back to the question of what tooling would your medical writer be using? Particularly if they are willing to edit a QMD file to begin with, they can write all of the narrative. I can imagine wherever we had inline code, they could put blanks and maybe leave you a note saying, this is what number would go in here. And then maybe some placeholders for this is what figure would go here.

And then all you need to do is sort of grab that and in a way, fill in the blanks. Fill in the blanks with a lot of work, a lot of code that generates the figures and sort of tables that you are creating, but they can do that. I can also imagine if this is a person you work with regularly and you develop sort of a particular workflow, you can decide this is how we will always label figures. This is how we will always label tables. And in a way they could almost write the cross references to your figures before you even write the code to produce them because ultimately all you need to do is to grab that same label from the cross reference and use it in your code cell.

So can we demonstrate the use of hypothesis to add comments? So let's go ahead and give that a try. I'm gonna open up this.

Here is hypothesis. Let me go ahead and log in. I hope this was the bit that I was nervous about.

Okay, how about we come back to that in a second?

Let me answer another question and I will come back to this in a second.

All right, so another question asks, so the QMD binder versioning replaces packages such as RM? Not necessarily. So there's still a place for RM here. RM would be useful for, for example, if you are using a, if you're collaborating with somebody else, if you're collaborating with somebody else, having RM activated on that project would mean that when you sort of pull the changes and you start writing things and you're sort of working in that RStudio project, you're using a certain version of those packages. And if you update any one of those packages, say there's a new functionality you wanna be able to use and you update the log file that goes with it and push your changes, your collaborator can now pull and next time they are working on the document, they're going to be using the right versions, the same versions as you are for your packages as well.

What binder allows us to do is not two people necessarily both authoring code locally in something like RStudio, but creating a virtual environment where someone might write the code. It's entirely possible that something like binder could be a essential component of your collaboration as well, personally for me, that's not something I've used before. That's not to say it's wrong in any way. But what I do like about something like binder, this virtual environment is for sort of the passive,

for the audience of your paper, the reader of this paper who might be doing it passively, it provides a great way of being able to be like, let me just take a look to see how they made this figure. What if I change this a little bit? What would that look like? So I think it's useful for that experimentation.

I've personally not used a virtual environment as such for like actual development of the manuscript, if you will, but it could be as well.

All right, let me go ahead and stop sharing for one second to see if I can reset this thing while I'm answering other questions.

Okay, all right, okay, maybe I can't do this, sorry.

Okay, maybe I can't do this, sorry. There's, I guess that's the extent of my multifunctioning. I apologize, but I won't be able to reset my password to be able to demonstrate hypothesis.

Let's see where we left things.

Answering audience questions

For the online version, I can have interactive figure or plot to embed, but it's difficult to do that for PDF. Sure, so for a PDF, you wouldn't be able to have interactive figures. I think that this speaks, this comes back to sort of the environments where we publish.

Many journals still want a static PDF. Ultimately, there's good accessibility and portability reasons for that, obviously, but it does limit what you can do. However, I can imagine a situation, assuming the reviewers and the editors are okay with it, that has a static version of a figure on the PDF, but because it is quite now easy to host the paper online, and if your journal allows you to have a version of the hosted version of the paper online as well, you could link to the online version, the online hosted version of your paper, and there you could easily have the interactive version of the figure.

How can we get cross-references and captions for Table 2 with spanning headers, so using GT instead of Knitter Cable? So you can use any sort of, the tables can be produced with any package, ultimately. So if you have a way of making a pretty table with GT, and I guess I will say if either your output is an HTML or whatever table you're trying to produce also renders nicely to PDF or Word or whichever version, whichever formats you need as an output, you should be able to sort of use that.

The caption itself, so anything like spanning headers and whatnot, those are all going to be in the R code of the, in the code cell, so in the portion where the R code lives. However, for the caption of the table, you are going to want to use that as a Quarto cell option.

So let me share my screen one more time if I can find the zoom. Here we go. Share the screen one more time, and let's take a look at where we had a table. So the only bit that is Quarto specific is the caption, but anything else that would be, so in this case, this is using the GT summary package, but it could be using any other package. You basically can do, create your table however you want there. So I would say in a way it is a lot more sort of like versatile than just using Knitter Cable where you would put the caption inside of that as well.

Okay, let's see if there are other questions I may have.

Okay, so a follow-up to the interactive figures question. I cannot convert because interactive plotly figure. No, correct, that you cannot have an interactive sort of figure in your PDF document. So that is, I don't think that's in any way a user error. That's not feasible to do.

Okay, I think that we try this one more time to see if I can get it right this time.

Okay, I'm sorry, no hypothesis for now, but hopefully if you go to the link for the manuscript and you are able to just create an account, you'll be able to play with it yourself. So playing around with hypothesis does not require just me sort of showing that to you. You can also go to where the manuscript is hosted and log in and actually interact with it yourself as well. And hopefully it will be self-explanatory.

I know that there was a question around the links. I wonder if anyone was able to copy them or if the platform we're using simply does not allow copying links. Is that the case?

If that is the case, maybe just going to this, if you can just type this in your browser, it's not as long. So it's just mine.porto.pub slash Porto manuscripts are med. That will take you to the slides and all of the links are on the very last slide. So you can navigate to them easily.

Mine, I can also add those links into the art consortium's webinar summary page. That way they can just look at it in there. I'm just going to go ahead and go to the art consortium's website so they can take a look when I upload it.

There was also a request to type them in the Q&A window. So I'm going to go ahead and do that. I don't know if that worked. But I tried.

Okay, well, thank you very much. Thank you, everybody. Bye.