image: thumbnail.jpg
Yu-Hung Chang, Phillip Lear & Brendan Scully | R in Manufacturing & Consumer Products | RStudio
Pratt and Whitney - Brendan Scully
Kellogg Company - Phillip Lear
AGCO - Dr. Yu-Hung Chang
Abstract #1
In this talk, Brendan will showcase two of the tools developed at Pratt using R and hosted on RStudio Connect: a forecasting model for repair shop performance and a real-time dashboard for work management on the repair shop floor. In addition to these tools, he will talk about how to approach some common development and implementation challenges for decision analytics tools at large companies like Pratt.
Bio:
Brendan Scully is the Decision Analytics Strategy Lead for Pratt and Whitney’s aftermarket operations business. In this role, he is focused on how decision analytics tools can be used to improve the maintenance, repair, and overhaul of aircraft engines. Before this, Brendan helped other companies make the most of their data as an actuarial consultant at Milliman and Deloitte, and as data scientist at the African Leadership University.
Abstract #2
Phillip will discuss his organization’s journey in transitioning a set of mixed analytic technologies to using a standardized analytic content platform. This talk will offer real life lessons learned in overcoming organizational barriers, getting stakeholders on board, and running a successful POC.
Bio:
Phillip is a Principal Data Scientist at the Kellogg Company working with global sales, marketing and ecommerce teams. He also teaches R to economics students at Grand Valley State University and sits on the Advisory Board for the Economics department. He lives in Kalamazoo (yes there really is a Kalamazoo!) with his wife Jamie and their two corgis, Penny and Winston.
Abstract #3
Dr. Yu-Hung Chang is an Advanced Analytics Specialist at AGCO. She’s going to showcase an interactive app developed at AGCO using R and hosted on RStudio Connect, which provides comprehensive insights for global manufacturing quality analysis. The real-time dashboard provides not only forecasting failure rate based on different levels of parts, but also varieties of diagrams of cost and claims in different aspects. In this talk, Yu-Hung will share what are the challenges for development of analytics tools and how to establish some common development in agricultural manufacturing world.
Dr. Yu-Hung Chang is an Advanced Analytics Specialist working for AGCO’s Global Field Quality group. Her multi-disciplinary background combines technical fundamentals with leading-edge research and big data applications, using the most advanced technologies. She has authored numerous articles, published in inter-disciplinary journals, covering topics from Statistics and Computational Physics to Aeronautics and Automotive.
Transcript#
This transcript was generated automatically and may contain errors.
Everybody, thanks for joining. I am super excited and happy to have this great turnout.
So I don't wanna eat too much into our speaker's time because I think what they have to say is gonna be really exciting and interesting. I'm on the customer success team at RStudio, which means that I work with our enterprise customers to help them leverage their RStudio tooling and make the most out of their investment.
I've been working predominantly with teams in the last year and a half in the manufacturing and consumer products space. And I think it's an interesting space to be in. And what I've noticed is whether you're creating, whether you're creating large machinery or airplanes, or if you're creating little widgets, the problems and the challenges and use case are all similar in this space.
And so what I'm hoping today is that you'll hear from these three different champions in the organization and hear that the types of problems that they're solving and the challenges that they're facing, I hope they'll resonate with what you might be encountering in your day-to-day. And I'm hoping that we can continue this dialogue both at the community site, have some great questions and discussion following today, and make this an ongoing and regular group where we can connect and network and ask questions of one another.
So with that, I want to first kick off over to Brendan Scully from Pratt & Whitney. Brendan is a decision analytics strategy lead for Pratt & Whitney's aftermarket business operations. I'm excited to have him launch us up. Brendan, do you want to get going?
Awesome. Thanks for the intro, Katie. I just shared my screen. Can I just get one confirmation that it's coming through? Looks good. Awesome. Yep, we got it.
Great. Then, yeah, I'll start with an introduction of myself. I've been at Pratt & Whitney for a relatively short time, about one and a half years now. I was hired initially as a contractor, so through a contracting agency, to develop a forecasting model, and from there transitioned to a role on the strategy team where my current focus is on the strategy side of decision analytics at Pratt & Whitney.
In this presentation, I'm going to provide an overview of two of the tools that I developed at Pratt using R and RStudio Connect, and then go into a few of the lessons learned in my strategy role that apply, of course, directly to Pratt, but I'm sure apply broadly to other large companies as well.
Engine repair forecasting model
So starting off, the first tool I'm going to talk about is an engine repair forecasting model. This is the first tool that I worked on at Pratt, and a brief background of Pratt & Whitney for those of you not familiar. According to Wikipedia, Pratt & Whitney is an aerospace manufacturer with global service operations, and, you know, what that means is we manufacture and repair military and commercial engines, and my role throughout my whole time at Pratt has been focused on the commercial aftermarket side of the business, which includes maintenance, repair, and overhaul of all of our engines.
So one of the biggest questions that we focus on on this side of the company is how many engines can our global network of repair shops repair in the upcoming year?
This question is important because the repair shops are giving annual and monthly repair goals that their performance is evaluated against, and throughout this planning process, any shortfalls that are identified then lead to a lot of the work that goes on throughout the year, from investing in new repair technology, capacity constraints, and making decisions to reallocate engines across the network.
This question is also a complex question. Repair performance and the number of engines that our entire network can repair is influenced by many interacting variables, such as the type of repair being performed, the turnaround time of each individual repair, the capacity, not only at the shops themselves, but within the different sections of each shop, and existing work progress and backlogs. So not an easy question to answer.
The current state, I guess before I move on to the solution, the current state that was in place to answer this question was largely done by taking the previous year's numbers and increasing and decreasing that amount based on knowledge of changes in the business, conversations with the general managers at all the repair shops. What lacked was a rigorous quantified justification behind these numbers.
And so what Pratt wanted was to make an answer to this question more robust and defendable by giving it a quantified justification.
So the solution that I developed that helps answer this question is a simulation-based forecasting model using R that has been hosted on RStudio Connect. So what this model does is it takes our global network of repair shops and models the flow of engines from the wing of the airplane through each stage of the repair journey all the way through the delivery to customer using a Markov decision process.
So the diagram on the right, I know the text is small and it's not really meant to be read, but it shows how the process is mapped out at a high level where each stage of the repair journey is given a state and then the flow of the engines from state to state is then governed by rules.
And those rules are all dependent on a large number of inputs from the turnaround time. So the capacity of the shops to the work in progress to the induction plan, to the engines currently sitting outside the shop and backlog. And so one thing this model allows Pratt to do now is instead of talking about it as a whole and doing all that thinking in your head, we can have focused conversations on individual inputs like the capacity of this section of the shop, has that increased or decreased? The turnaround times, are we expecting similar turnaround times on these repairs this year as opposed to next year? And then roll all of those inputs and conversations up to the model, which will then output a final number for engines inducted and returned.
So the main benefit is it offers a quantified justification for annual and monthly repair goals. And then it also allows for sensitivity analysis and scenario testing.
This has taken the shape of how does increasing the capacity of this shop actually impact the number of engines we get to repair as a whole. Or scenario testing, which came into play last year as COVID hit the airline industry. We had to rapidly change our goals and adapt to a new environment, new demand and new capacity as a lot of our workforce was reduced. And so this model quickly allowed us to run many different scenarios in a short period of time, which is something we weren't able to do before.
The last benefit is having the graphical user interface on RStudio so that ultimately I'm not the end user of this model and the end user of this model doesn't have strong background in R and can operate and use the model through RStudio Connect and the graphic user interface.
A few screenshots to show you what it looks like and what that interface is like. I'll highlight a few things here. And these are four of the main screens. The only thing not shown here is the ability to then download the raw data output for any analysis that isn't displayed here. But these four screens answer most of the common questions that Pratt is interested in, namely, what are the annual inductions into our repair shops and what are the annual returns? What's the comparison to plan? And where there is backlog, there's a page that then drills down and shows each individual engine that's being modeled, where the backlog occurred and what could be done to address it.
WIP management dashboard
The next tool that I'm gonna talk about is a WIP management tool. WIP stands for work in progress and it's the term we use to describe the engines currently in our repair shop.
So this model came about from, or this project came about from conversations with one of our specific repair shops. When an engine is received, a repair shop signs the contract with a promise date at which the engine will be returned to the customer. And the biggest issue at this shop was late deliveries. Their number one customer complaint was that promise dates couldn't be trusted and this impacted their brand. And so their question, their problem to me was how can the number of late deliveries, that is repairs taking longer than promised to the customer, be reduced?
I talked a little bit about the importance of this, but when an engine is not delivered to the customer on the date that they promised, it's difficult for the customers to plan their business operations. And they've told this shop many times, listen, it's okay if the promised date is later, but we just need to know a date so we can plan our operations. So that was one of the big important parts of this question. The other one is, of course, as Pratt misses delivery dates, it leaves a negative impact on our brand and reputation, which impacts our ability to win new contracts.
The other important part of this question is the issue that the shop faced was inefficient prioritization of their work in progress. Because with the backlog that they had, they found themselves focusing on parts and repairs for customers that simply screamed the loudest, which was described as a chaotic business process.
And so the fundamental issue here was a lack of visibility. For a complete view or understanding of what was in the shop at any given time, it required a manual pull from different databases and a manual report build. And because of the labor and time-consuming nature of this process, it was done once a day, and there still wasn't full visibility onto what engines were, what parts were on hold with the engineering team without going around and investigating on an engine-by-engine basis.
And so the fundamental issue here was a lack of visibility. For a complete view or understanding of what was in the shop at any given time, it required a manual pull from different databases and a manual report build.
The other issue that led to this problem was there was no measure of repairs expected delivery date, aside from intuition and judgment, knowing where in the shop each part was, but there was no quantified metric for repairs expected delivery date.
So the solution developed here was a WIP overview dashboard and forecasted repair completion date model. What this dashboard did is it takes real-time work-in-progress progress, so the real-time repair progress and hold data, and it aggregates it and displays it on the dashboard. So fundamentally, this is just a dashboard that displays the current state of the shop.
It has one analytical layer to it that takes historical repair data and uses that to estimate turnaround times and likelihoods of on-time delivery for current repairs. So one example here is a fan blade comes into the shop. Well, that fan blade with that type of damage has been in the shop 50 times before, and this chart here shows the actual historical turnaround times of part and repair, number A25343, so a repair that's been to the shop many times, and we have a good idea of what the range of turnaround times are. So it uses that distribution then to give an expected turnaround time, an expected delivery date, and then the likelihood that it'll be delivered on time.
Some screenshots from this tool. I'll highlight a few things here. The forward pane here that has those four boxes, that gives the direct overview in green on the right how many engines, how many repairs are in the shop right now, how many of them are on hold, how many of them cannot be touched because they're waiting for input from the customer or our engineering team, and then how many have missed the delivery date.
An important feature here is the bar chart or the histogram you see there. An interesting trend that this makes clear and highlights is how at the end of each month there's a surge in repairs that are due. So that first tall bar happens to be the last date of August when this screenshot was taken. And what this does is it gives, right up front, it gives the repair shop general manager and the cell leads a clear view of what's coming down the pipeline. And just that visibility alone helps drive the prioritization and the behavior needed to minimize those missed delivery dates.
Also included in this tool in the second pane here is a drill down into the individual repairs where information can be seen on their progress.
Decision analytics hierarchy of needs
So moving on to my role in the strategy team is I've kind of taken a step back from the individual tools and taken a higher level view and trying to work on ways that Pratt can prioritize decision analytics projects and best use them. And one framework that's helped me think about these projects is the decision analytics hierarchy of needs based loosely off Maslow's hierarchy of needs.
The idea here being each level of the pyramid needs to be addressed before the other one can even be worked on. And the way I've broken it down and the way I see a lot of projects at Pratt are data storage and organization is the first step and the most important step that needs to be addressed. Data relevant to your project or your business need needs to be stored and organized in an effective and thoughtful way.
However, storage and organization is not enough. There then needs to be data accessibility in place. So what's required to make the data accessible to stakeholders? That could be literal access, access approval, having the credentials or tools you need to access the database. But it also means is it stored in a way that is accessible? Is it in, there's a team here that has a lot of repair information stored in PDFs. Well, it's stored, but it's not accessible in a meaningful way.
The last step then is when you have successful data storage and accessibility is then building the analytics tools and then implementing data-driven decisions. So how can the data then be used to serve our needs and make better decisions? A lot of times there's a lot of talk and focus on the top of the pyramid without full understanding and focus on data accessibility and data storage. So this is a framework that I find useful to level set expectations for a project and then direct the right resources to the right problems before starting a project.
A lot of times there's a lot of talk and focus on the top of the pyramid without full understanding and focus on data accessibility and data storage.
Common challenges at Pratt
And then this last slide here are some common challenges that I've faced during my time at Pratt and I'm sure are not specific to Pratt. I also talk about some of the approaches that we've taken here to address these issues. They're by no means the right answer or even the only right answer or even complete answers to the problems, but they are steps that we've taken that have helped us.
So one of the biggest common challenges is lack of engagement from all stakeholders. And I see I'm running a bit short on time, so I will rattle through this somewhat quickly, but there's two main cases that we've identified or I've noticed here, and that's analytics tools developed without enough input from the end user. This usually means or results in the business needs not being addressed by the tool directly and ultimate ownership and integration of the tool into business processes becomes unclear and difficult.
The other type of issue of lack of engagement from all stakeholders is analytics tools being developed by the business users, in our case, the repair shops without involvement of the central technology team. This usually leads to fragile and fragmented architecture, manual data transfer, copying of Excel files and CSV files and an inefficient use of resources.
Two ways that we've addressed this and we are working to address these issues are to involve the business user more. For both of these, you need to approach these problems and these situations with the intention of bringing the stakeholders together. To involve the business user more, the design needs to be centered around the business need. Then effort needs to be put in to develop and implement a minimum viable product and then iterate and increase investment over time all the while incorporating feedback to meet the business need.
Finally, or secondly, to encourage the business users to involve the digital technology team, regular communications to understand business needs and capabilities should be established. We did this in the form of phone calls to all our repair sites to understand their needs and talk about our capabilities. And then the big thing here is organic involvement of the digital technology team through a series of successes. So have the business users organically come to the digital technology team because you've delivered value in the past.
The last issue is one that I'm sure many organizations face and that's inefficient allocation of limited resources. So there's limited funding and limited hours in the day. The obvious answer here that has existed but sometimes isn't always used here at Pratt is developing prioritization methodologies. So that's taking a step back, clearly defining strategic organizational goals and then designing a resource allocation process that stimulates and promotes conversation on how these projects align to the strategic organizational goals.
All right, I will stop there and pause for any questions. I hope I didn't go over time.
Oh, you were great, Brennan, thank you. I do know we have quite a bit to go through for an hour. So I may put some of the questions to the chat window initially, if that's okay.
Brennan, I know one question Jeff just asked is for data storage, what solutions or technologies are you currently using? If you wouldn't mind answering that one in the chat too.
Totally, oh, answer in the chat or answer it? Or we can answer that one quickly now. Okay, I'll answer that one quickly and then other ones in the chat. But data storage, the biggest tool that we use is SAP. And that has all of our repair data and really all the data that's needed, most of the data that's needed to build these tools.
The one interesting thing I'll comment on that is SAP is set up for us at least in a way that it is, all the resources at SAP are used to store the data and you can't connect directly to SAP. So we've developed other database tools that for the sole purpose of bringing the data from SAP on a periodic basis and then making that as an Oracle database, Hadoop is another tool that we've used to connect directly to SAP. So you can't connect directly to the data. The big reason there is it's resource intensive to pull from SAP. So you have one tool that does that and then that becomes the source for a lot of other tools.
Awesome, thank you so much, Brendan. For our second presentation, I'd love to turn it over to Philip Lear as well. So Philip is a Principal Data Scientist at the Kellogg Company.
And your share there, Brendan, there we go. And I think, Phil, you should be able to share now. And again, please ask any questions through the chat too and we'll follow up with the presenters after and I'll put that on community too.
Thanks, Phil. Can you see my screen and are you seeing the right presentation? Sorry, I'm using multiple monitors and I always get confused. Yes, we can see. What do you see versus what I see, okay. We can see it.
Kellogg Company: analytics history
So like she said, I'm the Principal Data Scientist at the Kellogg Company. I've assumed most people are fairly familiar with Kellogg, but just as a brief history, we are a 115-year-old food company. The slide kind of shows the timeline. What I'd like you to get out of this is that on the left, you sort of see these great brands and historic development of sales and marketing techniques and all that. That is sort of like the first part of Kellogg's history. Most of this probably took place before there were IT departments or, certainly analytics departments.
But in the middle, you start to see where IT starts to play an important role. So in the seventies, we acquire a frozen waffle company. It's very different than shelf-stable cereal. 2000, we acquire Poshy. This is an entirely different supply chain because it's natural and organic. New set of customers. 2001, we acquire Keebler, which is a giant snacks company in its own right. They actually use something called direct store delivery as their distribution model, which means the employees who are the people who drive the trucks and the trucks are assets and the logistics are managed. So the amount of data generated by a Keebler cookie from factory to the store shelf is significantly more than a box of Frosted Flakes would have.
We actually had to adopt Keebler's ERP system because what Kellogg had at the time couldn't handle it. Pringles is a massive global supply chain. In 2015, we expanded to Africa and Egypt with major brands. 2017, RX Bar is acquired, which is a new set of customers and primarily through e-commerce. And then in 2019, we actually spun off the cookie and fruit snacks and ingredients business of Keebler and sold it to Ferrero.
My hypothesis would be that the right side is sort of enabled by IT and analytics, but it's still a 115-year-old company and there were still a number of challenges we've had. I started in 2010, for, if you need some reference, TikTok by Kesha was the number one song. I honestly don't think I know any of these songs. I was surprised when I looked these up.
But our ecosystem at Kellogg for analytics, and keep in mind, we have SAP, we have all these other things, the reporting at the company was well-established, but analytics was sort of niche. Everything was sort of segmented. IT had its own thing. Analytics was primarily in the business, either marketing or sales.
We had our own network of servers, which by that I mean, we had desktops that sat under our desks and had RAID drives attached to them. And if a hard drive or a fan failed on one of these RAID drives, there were more than one occasion where I actually went to Best Buy to go buy a new hard drive. We had a little toolbox and we unscrewed the plates and pull the drives out. Very, very, very, very sort of guerrilla analytics team at the time.
The desktop SAS license, desktop R and Python, obviously. We could not touch SAP at all. So any data we needed to get from SAP traditionally had to go into a transfer server. And then it was sort of like this Rube Goldberg machine of how to get data from one place to the next. And then probably the worst part to me coming out of grad school was everything that was a deliverable was effectively in Excel or Python or Excel or PowerPoint, sorry. And I was an econ grad. I hadn't really ever used either one of those to be completely honest.
So in the first few years of my time at Kellogg, I think I wrote more VBA code than most people probably would ever write in a lifetime because they'd want simulations and some of these other things. The downside of all that was that if you build a tool in Excel and you gave it to somebody, there was zero chance of ever updating any of the data or any of the methods or anything in those tools.
Evolving the analytics ecosystem at Kellogg
So over the next few years, we really focused on how we got data. How we delivered sort of these data products to people. If anyone who I work with is watching this, they may disagree with how nice and neat these lines look, but it was certainly an improvement over what we had when we started. We'd moved to a SAS server, effectively let people share code and compute time and all these things across the network. We had SharePoint to sort of deliver PowerPoint in Excel to people versus sending mass emails. This helps a little in that we can replace a version on SharePoint, but obviously if someone copies, takes a copy, there's no way to update that.
We got a Tableau server, which I think reduced some of the PowerPoints, but people were stretching the limits of Tableau and what it could do. So we set up some .NET servers where people could build custom applications and tie them in to, short procedures could act on databases or you could run a SAS for procedure. It's hard to do these things, but there's a fair amount of bureaucracy you'd have to go through to get through these.
So in 2018, there was a push that I think what people said is cloud first, mobile first, API first was sort of an IT rallying cry. The majority of the analytics people
had been moved into IT by this time.
We had multiple servers. We stood up a number of database management systems that people could access depending upon the level of skill they needed access to. And in 2018, I was just sort of sent on endless amounts of conferences as we approached building the strategy. And there are a lot of them that were less valuable than others.
Just by happenstance in 2018, there was this conference called URL, which is the enterprise application of the R language. Apparently it's not normally in the US. So this was sort of a special occasion and I grabbed a couple of people on the team and we went to this and I loved it. I mean, it was awesome to hear people talking about using R and open source sort of in a production environment and enterprise environment. I think RStudio had a small table there and it was Thomas Mock actually, I remember this. And whoever he was with, he got grilled for like two hours by me and an engineer I brought and one of the other people on our team on what Connect did. Because we were effectively going through the idea of like what we were building to deliver this cloud-based architecture for analytics.
A couple of months later, RStudio conference is in Austin. We go to that as well. And again, Thomas Mock is there and I accost him and talk to him. And I told him I was actually skipping. I teach at a university, teach an honors class that Econ students effectively teach them how to use R in sort of real world applications for econometrics. And I told him I was skipping this and I was teaching. So they got a week off, but I was gonna come back and share all this with them. And he told me about RStudio Cloud, which was in beta at the time.
So I went back, end of January, I go back, I start teaching and I attempt to switch all of these juniors and seniors onto using a cloud platform. Remember, these are college kids. They don't really know, they're econ major college kids. They don't really know much about this stuff. So I had sort of one experiment going with this cloud-based analytics with my students. And then we came back, we started really looking into things. Prior to this, we looked at the usual suspects, the anacondas. And I think one of our engineers built Zeppelin which is Apache's sort of version of a notebook, but it's based on the Java runtime environment. I think it was the first time I ever saw someone sort of get booed when they presented something because the data scientists at the time didn't really wanna learn Java to run their notebook.
What worked and what could have been done better
So what I'm gonna go through now is sort of a story in three acts, two parts each act. Three things we did well and then three things we could have done better.
So the first thing we did is set up these test environments. Like I said, we tried multiple environments. We, I think we somehow provisioned some servers and started setting all these little tests up and doing things. Because as we were having a conversation with IT, which is a fair amount of IT people are like finance managers, program managers. Everyone was very, very interested in switching to open source, which is very odd to me. So I started and I talked about R, Python. Most of the time people thought there was some security risks using open source software. But somewhere along the line, there was a change of heart. I come to find out this is mostly because they thought open source meant free. Like they thought that switching to R would mean like total cost of ownership would go down substantially.
And it was very difficult to say, to explain to people why we would need this architecture, why we would want a product like RStudio Connect. So we just built our own version of, you know, as close to RStudio Connect as we could get. We built, you know, RStudio Pro server. We built a hosting environment for Shiny applications and Markdown applications. And we started converting some of the existing projects we had into R and deploying them through Shiny. You know, we set up, I think we had a Git lab that we installed on one of the servers and started, really built the entire ecosystem to show people how it worked and kind of prove out, you know, our vision of what this cloud-based analytic system would look like. And it was pretty successful.
Now, unfortunately, what we didn't do was sort of plan for the long-term management of this. So we stood up the RStudio server and we got people accounts. We had all this, we had the POC, we had project managers, we had all these things, right? And then once it was a success, everyone said, yeah, this is successful. And then they sort of handed the keys over to our team. And while it was, I mean, it was certainly interesting because me and another person on my team started actually approving, you know, approving the requests for licenses and setting them up on the server and, you know, writing back code to handle updates and all these things. That really wasn't the best time for our people.
So now we've sort of hired some people who are full-time managers of the system. They can, you know, the compiler needs to be updated. They can update the compiler if, you know, we need to expand or buy more space, whatever, they're sort of managing this stuff. It was a good problem to have. It became very successful and it sort of overwhelmed us in how much work needs to be done to maintain and add people. But in retrospect, it would have been ideal if we would have thought all this out ahead of time.
The second thing we did that worked really well was we went and found all of the analytics people throughout the company, not just data scientists, not just people in IT, people in supply chain, people in sales, people in the regional finance offices, the people who are doing sort of amazing things, often in something like Excel or Tableau. And we just gave them a count. We set up their connections they needed, gave them a brief overview of how to get started and let them go.
We went talking to a lot of the IT people with this idea, they weren't terribly into it. There's always sort of concerns for security managing access and doing that. But I think we knew it was the right thing to do to just get people using the tool. I attached myself to a project building a rather complex application in Shiny, sort of a crud tool that interacts with our promotional data.
Another member of our team actually built our, at the beginning of COVID, built the entire COVID tracking for our factories, which anytime an employee either obviously contracts COVID or reports that they've been in contact with someone, we needed to maintain this. Now, if you think back a year ago, sort of the confusion around all this, like built the entire tool, built some security for it, sort of did all this in Shiny. And I think this really sold everyone that this is the route to go.
And I think this really sold everyone that this is the route to go.
Now, unfortunately, we didn't have any user groups or any connections between all these separate people. So we quickly had a lot of searches on Google on how to do things and some inconsistency in the libraries people were using. And we were managing the library centrally, so we'd get all these requests for depreciated libraries and have to manage all these. So what we're trying to do now is actually have a group of people who meet to share what they do and sort of standardize some processes. Not that we don't want people to be innovative or anything like that, but there was just sort of a lot of work, managing people doing very similar things in different ways.
And then the final thing is we really, really pushed the idea of having a centralized analytics platform. That was cloud-based, that if you want to deliver content to someone, it was one place they could go to. It was web-based, everyone could access it. If you remember, it was sort of the push for the cloud, mobile API. Mobile last year sort of dropped off the radar. I assume it's because everyone was working at home, but the idea of the centralized platform on a cloud-based architecture really enabled us to go through 2020 without any real delays in the work we were doing because we sort of had this stood up right at the beginning. We worked through a lot of the issues sort of in the middle of 2020, but having this enabled us to handle all these new things.
Obviously we work in the, I don't know if anyone else is in sort of the CPG business, but, or I guess if anyone was shopping at a grocery store, you notice this, there were massive supply constraints. Most likely a lot of promotions were pulled that were planned. The way we allocated our production was really to fill shelves versus anything else. All this stuff was sort of built in RStudio and delivered through Connect. So everyone is sort of on board at this point.
Unfortunately, the people we didn't sort of bring into this initial cohort were being asked to build things in R without really knowing anything about R. So I think this is another problem of success sort of causing an issue. We're trying to rectify this now, we're really taking, Katie will know, I routinely emailed her throughout the year last year saying I really wanted to make sure we've got an education thing going. Entirely my fault that it never really took off. Now we're very, very, very committed to doing this. We wanna upscale tons of people, we wanna give people these tools, we wanna see what people can do.
Even today we're going through sort of an architecture change. I was talking to some consultants who are managing migration of some things from SAS to R and they sort of asked about scheduling, like batch jobs. And I took them through this idea of building an R Markdown file that runs on a schedule, generates a report, emails people sort of the changes made in the system. I think when people started to see this, I mean, they really see the power, now it's up to us to sort of make sure they have the capability. But I went really, really fast because I know we're behind. Hopefully that gives enough time. Please ask questions. You're welcome to email me or on LinkedIn, however you can get ahold of me if you wanna talk more. I left out a lot of details on the architecture because I don't think you're supposed to put your architecture up just in case there's any competitors. But if anyone wants to talk in depth, please let me know. Thank you.
Introduction to AGCO
Awesome. Thank you so much, Phil. Oh, Katie, I'll let you go. Sorry. Thank you, Phil. I think a lot of the challenges that you described are pretty common either in full or in pieces for a lot of folks. So again, I'll encourage folks to put questions in the chat window just for the sake of time because I'm really eager as well to have you hear what Yu-Hung Chang has to say. Yu-Hung joins us from AGCO and has a great app that she's chosen to spotlight with us today.
Hey, I'm going to share my screen. Can you see my screen now? Yes, we can see it. Yes. Okay, great. So.
So my name is Yu-Hung Chang. My background is actually in aerospace engineering and statistics. And after graduation, I had a great opportunity to join a fantastic team at AGCO. And now I'm the advanced analytics specialist at global quality field in AGCO. So today I will quickly go through, give you an overview about our company because that will help you to understand our challenges and the difficulties we are facing right now because I believe most of you have ever, probably everyone has ever had Kellogg's product, you know, their food or the flight, either Airbus or Boeing, you know, they both use present with these engines. So you sort of have your customer experiences with their products, but I believe probably no one has ever used our product before, I mean, neither do I. But I believe many of you have had the food, you know, either plant or harvested by our product.
So we are, we are a global leader in the design, manufactured and distribution for agricultural solutions. And we are also the leader and experts in global quality solutions. So these are the four big leading brands we have for different type of agriculture vehicles, including tractors, rogators, surrogators, and anything you name it. But we are also the only full one product in company in agriculture industry. Why I say that? Because we also has a very comprehensive
grain and protein department, which we own these five brands. So you probably know GSI and also CNBRA and other brands as well.
So our values and mission stand on, you know, providing sustainability and good quality products to farmers and also improving farm income.
So just a quick overview to our history. As you can see, Agco is a pretty young company. Last year was our 30 years anniversary. And we officially started in 1990. And in the past 30 years, actually we have acquired over 40 companies. That's why, that's one of the method we grow this company so fast and now become so big.
And during, you know, acquiring other acquisition and, you know, it means you have to also acquire that company's data vault as well. It means we kept gather other people's data vault into our own system. That make our data actually very dirty. I mean, more complex than, you know, the CFD turbine flow data I've ever handled before. And that actually caused a lot of challenges to us, especially for agriculture.
Even let's talk about tractors. Even you see different type of tractors doesn't mean we can really just compare them together because we have tractors designed for the maybe common uses. And also we have tractors designed for vineyard. We have tractors designed for grants or for fruit. So how to standardize our analysis tool to make sure when we analyze our data, when we analyze our product quality, we won't make mistakes like comparing apples and oranges together. And we can provide nice, you know, KPI for everyone, especially now Acco produce, you know, 133,000 vehicles per year. And we have 43 manufacturing locations.
And in the past, you know, some of the manufacturing locations were joined by acquisition. So it means each factories actually has their habits, their know-how. So we have to understand, you know, what they have and what they try to do. And also make sure when we standardize this system, we can balance everything. And we can try to make everyone, you know, not only our engineers happy, but also make our customers happy.
And by 2020, actually we have 3,275 providers and our products and customers are, you know, spreading worldwide. And we produce not only, you know, tractors, irrigators and different type of, you know, or grain storage. We also produce, you know, grain products as well.
Why R and RStudio at AGCO
So this is about Acco. And now we're talking about what do we do with R, what we choose R and what are our challenges? So one of the reason we choose R, or you can say we choose RStudio and RStudio Connect because it provides a very, to me, it's a very friendly environment and also very nice flexibility for us to customize any tools we want.
So our challenge is, you know, Labyrinth data science to supercharge global quality control. So let me use this map to go through our design map. So to us in data, you know, in data science field, our wonderland is the data pool with puzzles, which are, you know, the market dynamics, product qualities, customers' experiences and so forth. And our adventure starts at the moment we first exam our data and try to find a way out, right? That's what Alice tried to do.
And when we discover and explore our market dynamics, then we start to do our product design. That's the first phase of our R&D. And then we do tech development, try to finish R&D. And then once we finish up, we start to make products. And then we do test. And after test, we deliver our products to our customers.
And after a while, we start to receive feedback. And like all the companies we got, angel customers who love our products for all reasons. And sometimes we have, you know, just, you know, unhappy customers. Yeah, I won't say crazy customers, but unhappy customers. And when they find out, you know, that the products are not as good as they expected, they file plans or, you know, the product is failed too fast, we receive plans. So let's build better customer experiences. And our goal is to make everybody happy and try to improve our customer experiences.
R&D challenges in agriculture
So some people may ask, what do we do during R&D? Because, you know, one chain in quality control is R&D. And in automobile industry or in aerospace industry, actually in agriculture industry, we all use DOE, which is design experiment a lot. But for agriculture, most of the time we use that for like do some scientific way to do plants. But it's a little bit hard to use design experiment methods in agriculture machine building because unlike automobile, you can predict which environment your car may experience, right?
I mean, the road, we know the average quality of road. We know we have highway, we have just regular road or sometimes pretty muddy and we know traffic lights and we know the probably average driver habits. But for agriculture, we have very, very, you know, over varieties of geography characteristics. You know, the machine could be used for very, very cold environment or very, very hot environment or very muddy environment and also different plants. And we cannot just design one harvester for all type of plants. And also we have different climate.
So it's a little bit like too many elements to DOE plus when automobile industry do test driving for their new product, they can build their own campus and just do test driving. But to us, when we do our test driving, it's either plant something or harvest something. And when you harvest out the plant, it means you got nothing left to do more experiments. So that's one of our challenge and also dilemma. So it means for our product design and technology develop, DOE method is pretty limited. So it depends a lot in the, you know, plant analysis.
And when you harvest out the plant, it means you got nothing left to do more experiments. So that's one of our challenge and also dilemma.
Centralized tools and data solutions
So one solution to us is to develop a centralized tool, providing information and comprehensive insight of data analysis. So we have several methods and the first method is to standardize our performance measurements, such as claim rate and repair frequency. And everybody has to use the same methods to do the math. Otherwise we cannot put all the numbers on the same page. And also we have to use the same way to do cost and usage, our analysis. And also we have different type of KPIs to measure different type of things.
And the second thing is we recently start to integrate telemetry data, which means we can monitor our machines life. And we sort of, I think we receive data from each machine sometimes every five seconds, sometimes every 10 seconds. And what does it mean? It means we've got different type of code and when we receive the code, doesn't mean the machine failed. It could be something, it could be nothing. So we need to try to understand, you can say the language machines try to tell us.
And also we need to set up a threshold, like if we analyze telemetry data every single time, it means every 10 seconds is huge amount of information and maybe too much. If we set a threshold like, okay, we only get data like every 60 seconds, maybe that's enough. Or we have to do just every 100 seconds. So that's the part we are developing and try to figure out what's the best way to set up threshold code and focus on different brand, different machine and probably under different user environments.
And once we figure it out, we can build an alarm system to forecast failure. Means we notice, okay, some code keep coming out, means some part will fail out. So we can notice our farmer say, hey, maybe the O-ring in your engine, which part is going to, you need to replace that. Otherwise it will fail maybe in the next 10 usage hours, something like that. So farmers can replace that before the machine fails. So the farmers won't, got nothing to do and just with the time for his work.
And then the third thing is to build a predictive model to predict failure rate. Here we use Weibull analysis. So some people probably heard about Weibull before because it's a very mature method, has been applied in manufacturing industry for I think since 1970s. But in the past 50 years, actually Weibull analysis has been improved and modified into different forms. And we applied this method over 20 years ago and we kept modifying that. And in our latest tool, we built in one is you can say empirical method. Another one is based on maximum likelihood method because it depends on which way we collect the data and use some different scenarios.
And the last thing is other data source and the basic extensions. And like for every time we receive a claim, it comes with one global causal part. So one main part they'll sometimes come with very few replaced part or hundreds of replaced part. Are they related? Can we find out some patterns? That's also our job. We try to figure it out.
Major challenges
So the major challenge, like I said a few minutes ago, first is dirty data. Because during M&A, we kept receiving data from different companies and each company has its own habits to collect the data, to store the data. So we have to put that into our central ETL system. So when ETL received that, and then after a while, I got everything actually still very dirty. So we have to figure out a way to claim it without hurting important information inside the data.
And then is the limited sample size. And like I said, in agriculture, we won't be able to do that type of, a lot of tests like automobile industry do. So in certain way, our sample size is limited, but doesn't mean that the data size is limited. We got huge data size, just the sample size somehow is limited. And also we have unpredictable product usage.
Even if the same product, but when we sell it to different, let's say in the United States, but different states and different areas, different farmers have different usage preference. And also they probably use them for different purposes. So the product usage actually is a little bit unpredictable.
And also this tool we build has to be, work with big data and life data. We have to monitor telemetry data, congregate everything together. Also our users hope everything can be calculated in real time. So it means we need something to help us to organize everything. And every time we choose different filters and the filters has to be fully dynamic, we can get different results and the results has to be reliable. So those are our main challenges.
And that's the main reason we choose to work with R and also RStudio Connect, because after we build the user interface, we store everything on our SQL server. And all we have to do is send the message to, script to SQL and then tell SQL which part of the data we need. And if it's small amount of data, maybe just a few seconds, if it's a huge amount of data, maybe up to 20 to 30 seconds, but still acceptable. But another important thing is this tool can be used for,
by multiple users at the same time.
So before we start working with R, this is our first generation tool. We call AIDA, same name as the famous opera, you know, Princess AIDA. And this tool was built under Tableau and Tableau is a very beautiful tool for presentation, but we did have a lot of hard time to work with that when we, you know, the data got bigger, bigger and bigger. Plus it's not easy to integrate some complicated math algorithms to calculate everything in real time.
So eventually we just compromised. We pre-calculate a lot of things. So even everything we pre-calculated, you know, every day, but pre-calculation means we have very limited permutation combination. We cannot really get what user really want. So eventually we go to R and build this tool. We call that Analyticus. And this is a simplified version. And this is the main copy, as you can see. And I'm going to just give you a tour to Analyticus.
Demo of Analyticus
So this is the homepage and don't worry, you won't be able to use that because you have to have our VPN to access to this website. And this is just a quick demo to you. So, like I say, everything is dynamic, means if I choose, you know, different type of machine, they will go through and this is default. So it means we got all the manufacturing location together. And here, all I want to do, let's say tractors. I don't want, you know, harvesting, you know, different type of or implements or generators. And then we have a lot of manufacturing locations and this is my dummy data, one of my dummy data.
And I got so many dummy tractors here and let's just do, because I know we're running out of time a little bit, so just do something simple. And here I choose analysis level, you know, from global cost of part number and to failure area to failure feedback. And actually this is the highest level and this is the sub area and this is the sub sub area. And in the past, we usually just focus on global cost of part number, but then we figured out sometimes, you know, it's not about global cost of part. It's more about some area, just easy to get failed. So we try to, you know, use different way to figure out what's the prediction, sorry, the failure rate using different way to evaluate our Weibo analysis.
So first we got like data overall information and you can monitor your data by, you know, choosing this people table and this is our other indicator. And this is our alarm list showing us something like summary and we probably want to go from, for example, we always try to know what's our failure rate at our target hour. And previously we said our target hour is 1000 engine hour and say at 1000, the failure rate for these two already be 100% and probably reach this number way before 1000. And we usually want to keep this number, you know, as small as we want. And this is the full Weibo information. And the Weibo plot actually look like this.
Okay, so if we choose global cost of part number, you can see this one is about hose and we also want to see maybe we got, okay, that's only hose we got. And we got air pressure. We want to see what's going on here. And for air pressure, it could be, let's assume we got a lot of points here and only a few, you know, in low usage hour. We may just, you know, for some time, we probably want to see what's going on when we just cut off our data because some of them could be outliers.
And you probably notice the number change. And this is the total failure number also change. And the way we calculate the total machines covered by, you know, this global cost of part number sometimes also change because it has to be covered by the manufacturing date range. So beta is very important because if this number, you know, equal, I mean, larger than one, it means it basically worn out by time. So we always hope this, you know, can be, you know, larger than one. Okay.
Sorry, Rachel, I couldn't see, I see you raise your hand, but. Oh, sorry. I was trying to give a one minute warning in the easiest way possible.
And yeah, and also you can download data and everything very fast. And these are just more charts and like heat map, everything, and you can, you know, monitor things by different way and different region. That's it. And also one thing important to us is we can just download after, you know, selecting everything. We can also use the second layer filter to, this is a design to let them probably, they want to filter down a few things and they can download the reports. And this is the example of the report I already generated because I know we probably will, you know, go out of time.
Nothing like that. So overall, this is the tool we built. And actually there were a lot more functions for like predictions and analysis, but today we got a little bit running out of time. So I'm going to stop right here.
Q&A
Awesome. Thank you so much, Yuheng. It was great to see the application as well. I'm trying to scroll through and see if there were some questions there because I'd love to stay on and ask a few live questions to all three of our speakers, if anybody can stay on with us.
But again, we will post the Q and A and the link to the recording on the meetup page and on community.rstudio.com.
But Yuheng, one of the questions I see for you was, if there were alternatives to Webull that you were using, and Jeff just mentioned that he may have missed a few of those.
Alternative about what, sorry.
I'm not sure if Jeff is still on. Were there any alternatives to Webull methods that you looked at or did you just look at using Webull? Yes.
Because basically, well, actually, one thing I didn't have time to show is actually there are, we were focusing on different way to exploring Webull because like I say, there are several different ways to build a Webull data. But so far, we've tried like three different ways to build that. But other than Webull, we also try like, I mean, time-dependent Gaussian process, which I'm familiar with, and also other type of time series models.
There are varieties, but they all have to be, I noticed if I want to get a good result, I have to utilize a lot of telemetry data or you can say live monitor data because those data will provide not only the information from claims. Because when we receive a claim means something broke and we collect everything like usage hours and all type of situations of that certain machine. But live data also providing us information like that when the machine is still healthy.
So before we officially release other tools, we have to clear out telemetry data. And that's one of the challenge because currently we're facing first is the amount of the data issue. We try to find out what's the best way to set up a threshold. Otherwise, it's a little bit overwhelmed and probably won't make our predictive model sensitive enough. That's one problem. Yeah, so there are other issues like what code we receive from telemetry data we should use to get into our predictive model. So that's why some predict model we haven't fully explored and also we haven't released this tool yet.
Thank you, Yuhang. I think most of the questions to Brendan and Phil were maybe answered in the chat, but if there are any other questions that you'd like to ask them live, I'm happy to have you post them there or just ask out loud and unmute yourself.
Or if you're thinking of other questions, I will go for one more for Yuhang. I see Brendan had just asked, what were the reports you showed at the end shared directly with the customer, the machinery owner?
This is our inside tool. So the reports are to our PCR engineers and also our manufacturing factories. And for customers, as I know, we have another system working with Salesforce generating different type of reports. So some information we only show to our people. So, yeah.
Great, thank you. Are there any other questions that we missed?
Oh, one other question from Josh, Yuhang. Did you encounter any internal conflict or internal politics from using Shiny in place of Tableau?
Yes. We actually, I don't want to use the word fight, but we did spend quite a while to let IT department convince that Shiny is a good tool to be used because previously they really, really want us to stay with Tableau forever. And then we developed some cases, including I already spent time to build Weibo, actually two different type of Weibo inside Tableau to connect to live data and show them the turnaround time difference. And one's like five to 10 seconds the most. Another one is, forget about it.
So very soon they realized, you know, Tableau is good. We still use that for a long scenario. Just for this type of case, we really need to switch to Shiny. So yes, we did encounter some internal conflicts, but we solved it and now everybody happy.
So yes, we did encounter some internal conflicts, but we solved it and now everybody happy.