Summary
Machine learning is a data-hungry approach to problem solving. Unfortunately, there are a number of problems that would benefit from the automation provided by artificial intelligence capabilities that don’t come with troves of data to build from. Christopher Nguyen and his team at Aitomatic are working to address the "cold start" problem for ML by letting humans generate models by sharing their expertise through natural language. In this episode he explains how that works, the various ways that we can start to layer machine learning capabilities on top of each other, as well as the risks involved in doing so without incorporating lessons learned in the growth of the software industry.
Announcements
Machine learning is a data-hungry approach to problem solving. Unfortunately, there are a number of problems that would benefit from the automation provided by artificial intelligence capabilities that don’t come with troves of data to build from. Christopher Nguyen and his team at Aitomatic are working to address the "cold start" problem for ML by letting humans generate models by sharing their expertise through natural language. In this episode he explains how that works, the various ways that we can start to layer machine learning capabilities on top of each other, as well as the risks involved in doing so without incorporating lessons learned in the growth of the software industry.
Announcements
- Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.
- Predibase is a low-code ML platform without low-code limits. Built on top of our open source foundations of Ludwig and Horovod, our platform allows you to train state-of-the-art ML and deep learning models on your datasets at scale. Our platform works on text, images, tabular, audio and multi-modal data using our novel compositional model architecture. We allow users to operationalize models on top of the modern data stack, through REST and PQL – an extension of SQL that puts predictive power in the hands of data practitioners. Go to themachinelearningpodcast.com/predibase today to learn more and try it out!
- Your host is Tobias Macey and today I’m interviewing Christopher Nguyen about how to address the cold start problem for ML/AI projects
- Introduction
- How did you get involved in machine learning?
- Can you describe what the "cold start" or "small data" problem is and its impact on an organization’s ability to invest in machine learning?
- What are some examples of use cases where ML is a viable solution but there is a corresponding lack of usable data?
- How does the model design influence the data requirements to build it? (e.g. statistical model vs. deep learning, etc.)
- What are the available options for addressing a lack of data for ML?
- What are the characteristics of a given data set that make it suitable for ML use cases?
- Can you describe what you are building at Aitomatic and how it helps to address the cold start problem?
- How have the design and goals of the product changed since you first started working on it?
- What are some of the education challenges that you face when working with organizations to help them understand how to think about ML/AI investment and practical limitations? What are the most interesting, innovative, or unexpected ways that you have seen Aitomatic/H1st used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on Aitomatic/H1st?
- When is a human/knowledge driven approach to ML development the wrong choice?
- What do you have planned for the future of Aitomatic?
- @pentagoniac on Twitter
- Google Scholar
- From your perspective, what is the biggest barrier to adoption of machine learning today?
- Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers
- Aitomatic
- Human First AI
- Knowledge First World Symposium
- Atari 800
- Cold start problem
- Scale AI
- Snorkel AI
- Anomaly Detection
- Expert Systems
- ICML == International Conference on Machine Learning
- NIST == National Institute of Standards and Technology
- Multi-modal Model
- SVM == Support Vector Machine
- Tensorflow
- Pytorch
- OSS Capital
- DALL-E
[00:00:10]
Unknown:
Hello, and welcome to The Machine Learning Podcast. The podcast about going from idea to delivery with machine learning. Predabase is a low code ML platform without low code limits. Built on top of their open source foundations of Ludwig and Horovod, their platform allows you to train state of the art ML and deep learning models on your datasets at scale. The prediabase platform works on text, images, tabular, audio, and multimodal data using their novel compositional model architecture. They allow users to operationalize models on top of the modern data stack through REST and PQL, an extension of SQL that puts predictive power in the hands of data practitioners.
Go to the machine learning podcast.com/predibase today to learn more. That's predibase.
[00:00:57] Unknown:
Your host is Tobias Massey. And today, I'm interviewing Christopher Ngoyan about how to address the cold start problem for ML and AI projects and the work that he's doing at Aitomatic to help address that as well. So, Christopher, can you start by introducing yourself?
[00:01:11] Unknown:
Hey. Thanks, Thomas. I am currently CEO and and cofounder of a company called Itomatic. So that's just like automatic except it starts with AI. Our specialty, our software, help translate human knowledge, domain knowledge into machine learning models. It's quite interesting. Right? Specifically for industrial companies where that the problem of not being able to take advantage of human knowledge is the greatest.
[00:01:35] Unknown:
And do you remember how you first got started working in machine learning?
[00:01:39] Unknown:
No. I've had a very long career. I started in Silicon Valley back in the early eighties as a child. I was a hacker, you know, even the days before the Atari 800, if you can believe it. So I've I've worked the entire stack from transistors all the way up to machine learning algorithms and so on. But I think the first application of machine learning statistical analysis, and so on was when I ran statistical arbitrage trading company in Asia. I spent 13 years back in Asia between, you know, the late nineties and and the early 2000. Back then, compute and data was not as plentiful as it is today, but certainly applying data to win on Wall Street. Right?
The d e shawls of the world, and I took the ideas there and and applied to Asian markets and, you know, created what's called a market neutral trading company. You know, if if you work on Wall Street, you understand these terms. But it's kind of typical that financial industry that and, you know, on the other extreme, the porn industry tend to be the ones that take advantage of technology the earliest because that's where the big the biggest profit margins are, I suppose.
[00:02:49] Unknown:
In the overall space of machine learning, there is this question of the cold start problem or the small data problem. And I'm wondering if you can kind of describe a bit about what that is and how it manifests and some of the ways that it impacts the ability of an organization to even be able to start thinking about investing in machine learning or what their machine learning capabilities might be.
[00:03:12] Unknown:
Yeah. So the cold start problem can be thought of as and anybody who is trying to apply machine learning algorithms into their actual, you know, work, into the actual applications, will run into this issue at some scale or other. Right? And machine learning, as you know, the food for machine learning is data. And so recently, there's been a recognition that data is not plentiful. Data is not as available as we wish it to be. And I think that's generally known and accepted now. I think what's much less appreciated is that there's a very distinct scale of that problem.
Meaning, everybody run this through data issues. But for a large class of problems, the solution is, say, well, go collect more data. You know, wait a a week or 2, wait a month or 2. But also for this $25, 000, 000, 000, 000 industrial economy, that answer is not good enough. It doesn't work. So I think generally, the people in the field understand that problem, but I think there's still not quite an appreciation for the size and scope and distribution of that challenge as applied to different segments of the economy.
[00:04:22] Unknown:
And then as far as the specific kind of application or architecture of the AI approach that you're taking, what are some of the ways that that can impact the scale or quality of data that you need in order to be able to even get started, where maybe if you have a certain data set, you can use that to bootstrap an ML model. And then once you have that in operation, it gives you a path to be able to feed new data sets in to kind of get the flywheel moving so you can continue investing.
[00:04:51] Unknown:
Yeah. I'll give you an example at 2 extremes that I have worked in. 1 is Google and the other is Panasonic. So when I was at Google in the early 2000, I was the first engine director for Gmail and Calendar and so on, but became Google Apps. And broadly speaking, data is very plentiful, right, for a company like Google. If you don't have the data that you need, you write 10 lines of code, you conduct what's called a percentage experiment, you launch it out to some subpopulation of your user in the morning, and by afternoon, you've got a 1, 000, 000, 000 examples. Right? There's no surprise that companies like Google, Facebook, Twitter, and so on. Is what I call digital first companies are also the first to be able to take advantage of the powers of machine learning. And then at the other extreme, like Panasonic, which acquired my last company, back in 2016, 2017, we basically went in there, you know, very gung ho trying to apply our AI machine learning and and hit the wall very quickly. And the wall could not be easily removed by writing 10 lines of code and launching it out to production, because things there move at physical speed. Right? And manufacturing, line, robot arm on that line, or fishing industry, and so on automotive, these things, you can't just go and conduct these experiments and say, Okay, I just want to, I need a 10, 000 example of car crashes. It doesn't exist. Right? You can try to simulate it, but it still moves at a certain speed.
So there began, you know, I sat down and I thought about it. There's a fundamental pattern here, sort of what I call digital first versus physical first types of companies. And so industrial companies, again, which still powers $25, 000, 000, 000, 000 annual economy, the techniques and the systems and the approaches that we have from Silicon Valley don't readily work. And so there needs to be a different approach, an augmented approach. And as you mentioned, you know, 1 of the approaches to do data augmentation at the data layer. Right? There are companies like Scale dotai and Snorkel and so on that addressing it at that layer.
A larger view is that we need to inject human knowledge, human domain expertise somehow into into that equation that the data sets that are coming off of the sensors on these systems are not enough. And so at the data layer, you can do that. But at the algorithm layer, there's much less awareness that is happening as well. And then that what I call the even higher layer, I call the something layer, where we combine human models, human knowledge models together with ML models seamlessly. That's also happening as well. Many companies like ours, you know, in in the context of a Panasonic or for Bruno or Tesla and so on, are applying it quite successfully. And, again, not just at the data layer. I'm also interested in understanding kind of what the
[00:07:43] Unknown:
scale of information is that you need if you're doing, you know, a deep learning model that is typically very data hungry versus a maybe, statistical or a stochastic model that you can, you know, be a little bit more granular in terms of how much information you want to use and how you want to kind of scale the model and some of the ways that you're able to maybe bootstrap that process by starting with a simpler model and then kind of ratcheting up to a more complex or sophisticated model architecture?
[00:08:13] Unknown:
The real wisdom is the right tool for the right job. Right? But I'll take you to an extreme, right, to see where you cannot have, you know, 1 tool for all jobs. Take the problem of predictive maintenance, right, in a physical industry, like predictive maintenance of refrigeration equipment, right, or predictive maintenance of aircraft engines. Invariably, in our industry today, you know, and you talk to a lot of people, and you say, hey, you guys doing predictive maintenance? Anybody who says yes will invariably say what they're doing is anomaly detection. Right?
And turns out anomaly detection is not a complete solution to predictive maintenance. All anomaly detection does, it takes in large amounts of sensor data, and then it can then you know, after 3 months worth of data or so, right, you can say, okay. Now I'm seeing it. Today's example does not look like what I've been seeing the last 3 months or the last 3 years, but it does not diagnose what's wrong. Right? In order to diagnose what's wrong, if you wanna take that to sort of the pure machine learning way, you need lots of examples of failures. Right? But by definition, you don't have a lot of examples of failure, particularly once you stratify it, model year, you know, by workload, you know, by operating conditions and so on. Really, there's, you know, whether it's deep learning or, you know, more statistical approaches or even small data machine learning, if you will. Machine learning itself, by itself, is not enough to solve those those problems. And what you really need to do is that okay. But a human with 30 years of experience looking at the situation, look at the sensor data, and so on, Not just from that dataset, but also from their years of experience can diagnose that and say, in about 2 weeks, I think this compressor is going to fail. So there needs to be an approach where you actively incorporate that human knowledge, not just at the data level, certainly at the data level, but also at the algorithmic level. And then the question becomes, did you go back to the old expert systems? Right? But that seems like a a step backward. The real thing is, are there opportunities to combine that seamlessly with this new machine learning world? And that's what companies like ours, IcoMatic, and a number of others are advocating.
[00:10:24] Unknown:
And so in that question of being able to incorporate the human in the loop and have their knowledge and experience and expertise feedback into kind of the model development and model design process, I'm wondering how you're able to kind of bridge the gap between the skill sets that are necessary to be able to design and build the model and the person who is maybe an expert in, you know, maintaining large diesel engines or aircraft engines and managing kind of that communication path because, you know, if you're a machine learning engineer working at your laptop all day versus a, you know, aircraft mechanic, you're probably not gonna be working in the same physical spaces. And so just being able to figure out how do you manage that kind of communication path and the collaboration flow across those kind of disparate roles and responsibilities.
[00:11:15] Unknown:
At minimum, 2 things are needed to get into this future. Right? As people say, the future has arrived, it's just unevenly distributed. Right? So look what companies like ours have done in the case of Panasonic and Furuno and other companies. So 2 things I need. Number 1 is that we we can't go back to the slow, you know, workflow heavy human in the loop. That's sort of pure and actual human sitting there in the loop. Right? We have to automate that. Right? There has to be a way to operate at machine speed, but with human intelligence. So that's sort of challenge number 1. But related to that challenge number 2 is as as you build a tool to do that, that tool has to be better than the alternative.
And the alternative is, you know, expect for sitting down and telling you or me to say, well, this is the rule. And then we encode that using, I thought, or whatever into the it has to be better than that. Right? So 1 approach that we are taking at Itomatic is interestingly taking advantage of the latest machine learning technologies. Right? You know, you have the emergence of these large language models. Right? Which and I'm gonna use an analogy. There's a lot of people disagree, but that's a different debate. These models seem to have a knowledge of the world. Right? So instead of having to show a million examples of, say, a circle. Right, you can just refer to the concept circle. So in other words, once you're able to communicate with these models, using natural language, then domain expert can sit down and say something like or type something like, if the pressure has been rising in the last 2 weeks, and the temperature remains constant, then you should take a look at the compressor.
Right? A natural language, literally, a phrase like that somehow gets easily converted into code that gets easily converted directly into a model that can then be productionized. So that's the challenge of, you know, a company like ours saying, okay, we're gonna translate human knowledge into ML models for you guys. In terms of that kind of progression of
[00:13:16] Unknown:
the advent of some of these kind of frontier models and large language models and kind of foundational models with transformer architectures and then being able to parlay that into a system where you can use some of those natural language exchanges to then generate subsequent models. I'm wondering how you see that kind of progression in the machine learning space of enabling that and where it is, you know, trending towards in the near future, and maybe, you know, contrasting that with the approach of kind of traditional software development where we are kind of building on the shoulders of giants with progressive layers of abstraction and maybe some of the potential pitfalls or risks that you see in the ML space as you kind of chart that same trajectory?
[00:14:02] Unknown:
I love how you think about that because the answer is embedded in the question. Yeah. You may not I think you asked it that way precisely because it is moving in that direction. I'll talk about the long term first and then say, you know, come back to the short term. What are we gonna do there? But what's implied, or at least I infer from what you're saying is, I'm gonna put it this way. Machine learning based purely on data is not the future. Right? Once you have these models, you should be building on the shoulder to the giants. Right? Machines will increasingly understand more than just raw data. So there's a near future where the training and, you know, people refer to it, you know, as prompt engineering, but I think it's much deeper and much broader than that. Right? We will be able to train machine models, right, with knowledge, because they already possess some knowledge. And then we sort of just ladder up. So the future of machine learning AI is really increasingly, let's use the word intelligent, increasingly intelligent machines or systems that can understand, right, 1st, raw bits and byte, pixels, and so on. And then next lower layer concepts, like circles and squares and so on, and maybe even a higher concept.
So then their ability to reach, you know, and converse with us. Right? Whether it's through audio or to type language. Their ability to take direction and get trained will be increasingly more powerful. Right? That's absolutely the future. So that's why when sometimes when I talk about these people say, you know, you talk about expert systems. Well, I say, well, you can live by expert system, but on steroids. Right? Because these machines themselves understand what you're telling them. In the near term though, you know, before we reach that laddering, then there needs to be the initial translation layer. Right? So what we're doing at Itomatic is really the large language models are being used as that API to do the translation, and translate it into a DSL. Right? And it's strictly typed and syntax and so on. And then from there, we go into an architecture that has a teacher, a student, and an ensemble. The teacher is the model that was generated from that translator.
The student is a standard ML, you know, data driven model, but the teacher is sitting there training the student, you know, with various, you know, system data. And then when you assemble them together, it turns out that human model and the machine learning model can combine to be the best of 2 worlds.
[00:16:24] Unknown:
Continuing on this kind of topic of being able to progressively build more sophisticated machine learning models because of the machine learning capabilities that already exist and being able to use some of those kind of foundational models to help generate some of the subsequent stages, you know, drawing that parallel back to software development and the software supply chain. There are a number of issues that have come up in more recent years because of the kind of level of scale and sophistication and adoption of the software systems that we're building in terms of things like supply chain security, you know, managing the kind of logical complexity of systems as they scale out, and, you know, understanding how to manage the kind of continuous integration, continuous deployment.
I know that there are a lot of those same problems that are being tackled in the machine learning space as well, but I'm wondering what you see kind of as some of the up and coming challenges that the machine learning ecosystem is going to have to tackle drawing on lessons from the past?
[00:17:28] Unknown:
Absolutely. In fact, that reminds me of a symposium that is coming up in November called Knowledge First World. And it's quite distinct from other machine learning AI conferences in the sense that it is not a research conference. It is a practice conference. Leaders from government, there's a major general from the NSA coming, right, from academia. Jeanette Wing, former head of Microsoft Research and the current EVP of Research at Columbia and of Industry, you know, Panasonic folks, leaders, they're basically gonna go then compare notes. And the notes that they're comparing certainly include, you know, successes and failures in the application of these technologies.
But because these are, you know, life critical systems and processes, a lot of us in Silicon Valley and the digital economies, we have not yet directly dealt with. So for example, automotive technology, avionics, and so on. The ethical, the safety issues are ever present in their mind, even from day 1 before they build the first system. It's not it's not a, you know, afterthought. So a lot of those issues occur in those forum. The difference is that even even at present, these tend to be separate discussions, right? You go to ICML, and you talk about algorithms, and then you go to an event organized by NIST, and they talk about trust and human safety, and the expertise do not overlap. So I think there needs to be this conversation like this knowledge first world symposium where the 2 topics are discussed pretty much by the same people informed, you know, by the same systems.
I think there'll there'll be a lot more of that driven by this industrial economy far more so than the digital economy because when you make a mistake with, you know, a a digital app, maybe the customer, the user clicks on the wrong ad. But when you make a mistake with with a self driving system, then somebody dies.
[00:19:22] Unknown:
To that point of kind of coalescing different disciplines and coalescing different concerns, as you are incorporating humans in the loop of helping to generate the models that are going to power your operations, power your applications and business. Humans aren't inherently limited to a single kind of mode where you're not necessarily thinking in terms of, oh, I'm only going to be working in natural language, or, oh, I'm only going to be working in computer vision, or, you know, I'm only going to be working on time series. As humans, we experience all of those things simultaneously, so I'm curious how that question of kind of multi model development and model training factors into the ways that you're thinking about human first AI development and the need to be able to
[00:20:12] Unknown:
holistic operating whole? That is essentially the gap between the present and the future that I alluded to. Right? So at present, as a tools company, right, we rely on what's available out there. We're not here to build a large language model, we're to use a large language model to actually do something, you know, very useful and powerful for the industrial companies. So the mode that is available to us today is that natural language mode. But you're absolutely right. There's already people saying, you know, can I just talk to it? Can I sit there and, you know, record something, and then you take care of it, Of course, then you can just do a transcript and so on? But vision. Right? How do you take advantage of what a fisherman sees? There's a use case we work on where, you know, you think about how a fish gets from the ocean to your table. There's a company called Furuno, which is a global marine navigation giant, and they have what's called a fish finder, both for the enthusiasts, possibly like you and myself, all the way to large fishing vessels.
But imagine the ultimate future when you shoot down the sonar beam, and it comes back pictures of schools of macro and so on. Today, that's not the case. Today, it comes back as an echogram. And it turns out, even the people who build these systems are not experts at interpreting those echograms. You think of it as ultrasound, but much messier. And it turns out these fishermen, in this case in Japan, and there's like a 150, 000 of them, and each 1 is expert diagrams, and they said, well, okay. Well, that's clearly a bunch of sardine, and and I'm looking at it. I'm saying, that's just, like, red with blue. I don't know what that is. Right? So capturing what they see and then translating that to, essentially, this video that you and I would see as a fish is an enormous challenge. And that's something that we're working with Furuno to do exactly that. So that vision. Right? But eventually, you know, taking advantage of human knowledge and human expertise, the experience that is inside the expert's brain, after all, came through through these modes. Right? Through what they hear, through what they feel. 1 more example as I talk about it. On a manufacturing line, as we work on predictive maintenance for these equipment, I hear of a line manager, every morning, he would walk down the equipment line, and he would place his hand, right, on the machine that's just being turned on, and he would say, take this 1 down for maintenance, and the other 1, no, that's okay. And so there's something that he's feeling from the vibration or whatever of the machine as he walks by. That should be distilled and captured and scaled because there are not too many of these experts around anymore.
[00:22:49] Unknown:
And that also brings in the interesting question of the modality of touch, which is something that is, you know, still very much in kind of the early days of being able to say, like, how can we teach machines how to actually experience that sensation of touch and texture and feel versus just measuring vibrations because you happen to have a sensor that, you know, is looking at the vibrational frequencies.
[00:23:11] Unknown:
I know exactly what you mean. We have 1 project is to, you know, detect it. Again, predictive maintenance, this time of robot arms on the manufacturing line. And we can't go there and disturb, you know, the thing that's happening. So we use, believe it or not, EM signature. Right? Electromagnetic waves. So you use this loop, and you sort of put it on the arm. And as it's operating, you get an EM signature off of it. And that is 1 mode. Turns out that is not something that humans can perceive. But on the other hand, the human touching that robot arm will feel the vibration in certain ways. And those are 2 different data sources. Right? To machine learning algorithm, they don't look anything like each other. But if they can be fused and combined somehow in in a multimodal or architecture, then I think we can, you know, emulate or copy from that human expert.
[00:23:59] Unknown:
And so digging into the platform that you're building, Itomatic, and the kind of goals of how you're looking to be able to attain this human integrated approach to machine learning and model development and bootstrapping machine learning models because of a sparsity of data. I'm wondering if you can just talk to some of the kind of infrastructure and design understandable to your target users.
[00:24:29] Unknown:
The key, as a matter of product design, is not to go in and say, I'm gonna revolutionize what you do. I said, no. No, please. Right? This thing is working. Can you just make it better? So the product goal is to produce what is essentially a machine learning model. Right? And then whatever system that you're using, whether it's MLflow, Kubernetes, and so on, it gets deployed the same way. But how it is the provenance of that is extremely interesting. Right? It is no longer only data streaming in and then running through, you know, TensorFlow or PyTorch and so on. But there's suddenly a new path, and that path starts with a human expert sitting down, perhaps typing in directly. Right? And at this point, the product articulation is that natural language, or working with an engineer, and the engineer would type this in. And so that pipeline is the only thing that is new and different. Right? It's a very powerful pipeline. It's able to take human input, and then outputs a model that works seamlessly with the entire system. That's our new term product rule. But, you know, the vision for the whole company is very much along the lines of you what you alluded to, which is, you know, how else can we take advantage of of human knowledge before it's gone?
[00:25:44] Unknown:
As you have been exploring the space of figuring out how to integrate the human experience and kind of human understanding of these industrial systems and translate that into a model training and model development process. What are some of the ways that your kind of initial design and goals and your understanding of the scope of the problem have changed as you have kind of gone from inception to where you are today?
[00:26:11] Unknown:
1 of the key things in that is that every use case is different. Right? So the idea is very broad. We actually have an open source project called human first AI, where we put a lot of these, you know, concepts and code out there. But in terms of applying it to industry, the ones that are most, number 1, sort of receptive and most resident at the moment, as I mentioned, are not the digital first companies. It's more the physical first company, the refrigeration, the automotive, the oil and gas, right, the fish finding industries and so on. And each of those has a different in specifics.
But the amazing thing is, as I talk to these folks, I don't have to sell them about human expertise. They actually sell it back to me. They said, we've been wanting to do this for a long time. We just don't have the tool to do so. Right? When we've been trying to do what is the the normative way of machine learning and collect data, and we find that it's not just that it's expensive to collect, it's also very long. And in many cases, like a machine learning prediction model. Right? You don't have examples. No matter how long you collect, You can collect that and suddenly the next model year comes in and you start from scratch again. So but coming back to your question, each case, each use case, you know, has to be sort of thought through very carefully.
And that's why we are more of a tools company. We're not a consulting company. We have gained some domain expertise simply by working with the customers. But what we're really doing is we're enabling the AI engineers in these companies to work on these things. So the customers that we work with tend to be actually quite far along the AI adoption curve. They're not people who say, what can I do with machine learning? They have tried and failed. Right? And they say, I need some tool to fill this gap. On that question of
[00:27:59] Unknown:
even just understanding when machine learning is a useful approach or a useful tool, You mentioned that the companies you're working with have already established that fact, and they've moved on to, okay, this first try didn't work. How do we do it better? But as you do start to kind of grow your operations and you start to onboard new customers, what are some of the ways that you help them to understand that initial question of, is machine learning a useful application for my problem, and how do I start to think about framing this problem in a way that machine learning is able to help facilitate the solution?
[00:28:37] Unknown:
To be sure, what we deliver is through machine learning. Right? But it is a more broadened view of where these models can come from. In some sense, this is what professor at academics have said for a very long time. In the last 10 years, suddenly deep learnings kind of took over, right, and took all the oxygen out of the room as it were. And then a bunch of, you know, old cranks say, hey, you don't need deep learning for that. You can just use, you know, SVMs or other things. And I think that voice is sometimes lost. Right? And so, you know, we take a step back and say that AI is broader than just machine learning, certainly much broader than just deep learning. Right? And so the conversation we have is just like, you know, is the right tool for the right job? Would you like this additional tool? Right? And for many use cases, you don't need this tool. But for a vast industry out there, this tool is vital. And it's still part of their entire machine learning pipeline. The deployment takes place, you know, in standard AWS. And if they already have a machine learning pipeline, this works seamlessly with it. But it gives them extra sort of superpowers.
The main epiphany is, even for us, is, like, these companies have vast domain expertise. Right? They've been around for 20, 30 years. In some way, Silicon Valley, including myself, sometime we tend to go in and we say, you guys step aside. Right? Let me connect this up. Let me collect the data and I'll do everything that's needed. It turns out that doesn't work. I mean, I don't mean just culturally, but actually technically. So having a tool like this is not just technically pleasing, but it's also culturally appropriate for a conversation with these, both the executive level and and the line management level at these companies.
[00:30:22] Unknown:
As far as the adoption process for the organizations that you're working with, can you talk to what's involved in actually onboarding onto itematic some of the existing capabilities that are either useful or necessary for them to have and some of the ways that you think about the kind of tool chains and platforms that you're aiming to integrate with, whether that's the, you know, specific ML library that they're focused on or the deployment methodologies or kind of training capabilities, things like that.
[00:30:52] Unknown:
So the typical team that we work with, as I mentioned, is that they they already have something. Right? They've been trying, and somehow it leaves them, you know, wanting something that's missing. And they feel, are they crazy? Am I wrong? You know, why are other companies succeeding and we're not? What are we doing wrong? So we go in and essentially, we say, well, you're not crazy. You really want to incorporate that human expertise. Right? Not just because it is the thing that makes it possible, but we should go to market, right, in 3 months, as opposed to trying to collect data, perhaps even unsuccessfully or augment data for the next few years. So in that context, they already have something, they have a pipeline, they understand TensorFlow, they have Python programmers, they data science teams, and so on, PyTorch, whatever it is that they're using, the output of this tool that we have for them, this translator, is a machine learning model. Right? It happens to be an ensemble of a teacher and a student. And the teacher happens to be knowledge based, and the student is machine learning. But as far as the API is concerned, it's features in prediction out. That plays very nicely with whatever production system they already have. And, of course, as part of a whole product suite, we also have what's the what I just described is the what we call the knowledge first build, k first build to to build models.
A lot of times because of that, they also wanna deploy with us. So we have something called k First Execute, which helps with deployment management and so on. And And even, you know, looking at the operation, you know, out in the field, and then looking back and sort of improving it with additional data as it sits out there for 6 months and the model can be improved.
[00:32:35] Unknown:
You mentioned earlier that you have this HFIRST or HumanFirst open source project that is intended to encapsulate some of the same ideas and principles as what you're driving at with Itomatic. And I'm curious if you can talk to the relationship between what you're building at Itomatic and what the h first library exposes. Like, is it is h first a kind of core component to what you're building at Itomatic, or are they just kind of mirrors of each other where you're, you know, releasing some of the lessons learned from Itomatic into the h first project and they're not necessarily
[00:33:06] Unknown:
a direct dependency of yours? That's a great question. No 1 just asked me that. I can see a diagram that I should draw. So I talk a lot about, you know, this pipeline that starts with a human, you know, whether it's typing in Japanese or English, and so on. And then pops on the other side is a teacher model. Right? And then the teacher model by itself doesn't do anything yet. So it has to go into some target architectures. Right? So these architectures are architectures that anticipate the addition of human knowledge. Right? We talk about that the data layer, there's data augmentation, you know, companies like, you know, scale dotai and Snorkel and so on, that are doing documentation. At the modeling layer, there has to be some architectures that somehow combine this human model with the machine learning model, ensembles, and so on. Right? So to answer your question, all of those architectures that we're creating and innovating on and other companies, other teams as well. Right? If you look at K First World, the the symposium that I mentioned, Google Cloud AI is coming to talk about how they are architecting a human knowledge pipeline together with the data pipeline coming out with a with a back prop that essentially train the whole model. So lots of these interesting architectures.
So the human first open source project is a collection of all of these architectures. Right? And then also some of the use cases as examples of how they're being used. The translator, that is something that Itomatics is building. And other companies can build it too. But that that's not part of the of the human first project. So the the you can think of it as dividing it Human to model, that's Itomatics product. And then once you have this human expert model, what architecture does it go into? That's the open source project.
[00:34:54] Unknown:
As you have been building the kind of commercial entity and the focus there and iterating on the open source capabilities that you've embedded into the h first utility. I'm curious if you can talk to some of the lessons that you've learned at the boundaries between them and some of the ways that you have kind of factored some of the community knowledge and community experience into how you think about problem that you're solving at IDOMATIC?
[00:35:20] Unknown:
Your question can be thought of different ways. For example, you know, you can think of it as, you know, what is an open source project, and and how do different companies think about open source and and and so on. And then the other is, you know, how do you commercialize something that is open source? Maybe I'll think about your question that way and talk through it. The second part, because I've been in this industry, I've been in tech for for a very long time. I worked on the first flash devices that that is now all of our bodies and so on. At the time, we thought it was only automotive applications. There was a time when commercial open source was an oxymoron.
Right? CIOs say, I work with Microsoft. I work with Google. Right? But I'm not gonna rely on this thing that is sort of unfunded and so on. Today, that decision has completely flipped 180 degrees. People prefer to buy, people prefer to use open source. You know, if something happens, they're not encaptured by a single vendor. Even if the vendor that the company that is supporting that project, even that, you know, they go belly up, the code is still there. So in principle, they can still run their own thing. And the quality open source, of course, and I I don't mean across the board, but certainly some of the most popular project, the the quality is is pretty clear.
From a commercial point of view, I'll mention a good friend of mine, Joseph Jacks, runs this fund. This is entirely dedicated to commercial open source. In fact, it's called OSS dot Capital. The entire portfolio is an open source, commercial open source companies. So I think the economic model is well proven. In terms of how CEOs or founders think about open source, there's still a very big range. I'll tell you the wrong ways to think about open source. Right? 1 wrong way about open source is to think that is the funnel. It's a marketing funnel. Right? Oh, I'll just put this thing out there, and then a lot of people will use it, and then some of them will convert. The lesson of the last 10 years is that open source users come in 2 flavors, and they don't automatically transform 1 into the other. Right? The free people will always be free. Right? The commercial people are commercial for an entirely different reason. They choose open source, not because it's cheap, they choose open source, because it is, you know, available and and and, you know, they're not locked in, but they still want the support and the development from a commercial entity.
So for the founder, you know, it's it's I'm gonna write some code. I'm gonna put it out there, and, you know, I'll just start collecting money. Good luck. Right? So in other words, a company like ours, even though we have this open source component, and we truly believe in it, we put it out there for contributions and so on, It's still our responsibility to invest in marketing and sales team and consulting and so on to to go and win these accounts 1 by 1. Right? Because those people are not sitting there and saying, let me download. The open source availability does add to their confidence, but it is not the main reason why, you know, they choose to make the $1, 000, 000, 000 decision on this tool. Absolutely.
[00:38:22] Unknown:
The whole question of open source and business is always interesting to get people's opinions on it because it is 1 of those polarizing debates, you know, in the same category of tabs versus spaces or Mac versus Windows or the age old wars that will never die.
[00:38:39] Unknown:
Right. Right. And is it tab 4? Is it tab 8?
[00:38:43] Unknown:
Or e max versus buy? Or
[00:38:46] Unknown:
That's been long settled. Definitely vi.
[00:38:49] Unknown:
I was just gonna say e max and 4 spaces.
[00:38:54] Unknown:
I'll let you get away with 4 spaces, though.
[00:38:59] Unknown:
And so in terms of your experience of building IDOMATIC and the h First project and working with your customers and helping them understand how to encode human knowledge and human experience into ML models that allow them to scale their business and increase their operational efficiencies? What are some of the most interesting or innovative or unexpected ways that you've seen IDOMATIC and HFIRST applied?
[00:39:23] Unknown:
Maybe I'll talk it from, you know, the innovation. A lot of it comes from our customers. Right? So when we first started doing this, we were thinking, and it's still the case, many of the use cases is, you know, you have 2, 3 experts, the coaching, the refrigeration, the project that we work for Japan, you know, Japan, they're convenient store, but they're actually very large scale markets like 711. 711 in the US at all. Right? 711 is a large supermarket in in Japan. Family Mart, Lawson, and so on. So they have, in every store, they have lots of refrigeration equipment.
And there are 3 experts in the entire country of Japan that are available if you want to do this predictive maintenance, that can diagnose in advance what's likely to fail. So we tend to build these systems with companies that are running out of experts who don't have enough and to scale them. Then then we hear from a customer, the fish find use case. Right? And they say, can you help scale this? So the knowledge problem is not 1 expert, you know, 1 model, or 3 experts, 1 model. Can you take in the knowledge of a 150, 000 fishermen?
Right? Of course, we can just, you know, throw it out there and say, you know, go ahead and type into here, and then we'll generate a model. Could ask for fundamental question, what does it mean to integrate the knowledge of a 100 50, 000 humans? I don't know the answer to that yet. That's still a question in life. Right? But it does drive a lot of innovation, a lot of thinking among our research team. Do I just deploy a 150, 000 models? Right? And say this model is good for Hokkaido, and this 1 is good for Okinawa. But that's a very limited view. Clearly, there is something in the knowledge of these people that transcend geography, transcend, you know, day or night, transcend weather, and so on. But what does it mean to combine those things, you know, other than just a simple straight ensemble of these things in a way that's sort of optimal, right, in a way that we can have, you you know, can we have the intelligence of a 150, 000 people?
Right? Making decisions day in, day out. If we succeed at that a few years from now, that'll be a major milestone of accomplishment for me, not just for the company. Right? But these these are the thing that make me jump out of bed every morning and say, hey, I wanna go to do that. Right? Because, you know, you have this idea, you have this tool, and then your customers say, you know, hey, can you help me do this? And then it sort of changes the game of how you look at it. Another thing that we didn't dig into yet is kind of what are the
[00:41:50] Unknown:
natural limitations of this approach? What are some of the ways that you have to constrain the possibility space in order to make it something that is tractable for converting into a machine learning model where you obviously can't just, you know, download the entire experience of an expert fisherman or a, you know, expert mechanic at some industrial firm. And so just understanding how to kind of define and understand what are the appropriate bounded contexts in which you are trying to kind of build a target model and some of the ways that you are able to kind of understand the current limitations
[00:42:29] Unknown:
and start driving towards pushing those further so that you can kind of expand the range of capabilities and modalities that you're able to operate in. That is the essential difference between the next 3 months and the next 30 years. Right? I mean, a lot of us on Silicon Valley, the mistake we made is not that we're too late, that we're too early. Right? We tend to build things that make a lot of sense 10 years from now. But people buy things that make money for them 3 to 6 months from now. So you're absolutely right. What we do is, you know, 1 of the things that we want to be evaluated for, you know, is not that we, you know, human being come in, and then somehow we put probes on your head and then pull them a copy of you comes out. That's not what we're selling. What we're selling is we can do something that is much better than the baseline of your current approach.
Right? And typically, it's not 10% better. It has to be 10 x better. Right? It suddenly unlocks the ability to encode human knowledge, you know, that can do some of the heuristics. Right? It's certainly not better than human. It could be more consistent. It could be more scalable and so on. Right? But compare us against the baseline of not having it. And that, you know, when people take a look at that and say, oh, wow, I can go to market in 6 months, and I can slap this AI label, you know, on my system that actually does something useful. And I think that's, you know, that's enough reward for the next, you know, 2, 3 years to just to help company go to market and make a difference, a marginal, you know, an improvement on their product that otherwise would not be possible without this encoding of human knowledge.
[00:44:02] Unknown:
In your experience of working in this space and building your business, what are some of the most interesting or unexpected or challenging lessons that you've learned in the process?
[00:44:11] Unknown:
That it is harder to talk to Silicon Valley or say machine learning people about this problem than I expected. And conversely, it's a lot easier to talk to the people that are otherwise not machine learning experts. Somehow, we have this bias, and I'm partly responsible for it. So, you know, data will save us all. Right? Big data was was a big cry, just short 10 years ago. People are using this. And yes, it is in the in the future where you and I just talked about earlier, right, where data will sort of just ladder itself up. Right? Data will become knowledge, knowledge will become experience, hopefully, an experience become wisdom, and all of that is somehow automated. Right? But, you know, now it's where we see generally, but I I know that while I was still at Panasonic and talking to my friends, I said, hey, man. There's not enough data. They would say, well, just go collect more. I said, no. You is it not that simple?
That's 1 of the biggest surprise. It's no longer a surprise, but initially, I received the biggest resistance. Right? The admission that this is in fact a problem and that the problem worth working on, you know, from the very people that are the experts at these algorithms.
[00:45:20] Unknown:
Yeah. It's definitely funny as we reach what in different problem domains, what we think are kind of revolutionary capabilities or revolutionary approaches to solving a problem because of the fact that we're bounded by our view as engineers or, you know, programmers, data scientists. And then we talk to people who we're trying to solve the problems for, and they say, oh, I thought it already worked that way.
[00:45:44] Unknown:
Yeah. That's right. Yeah. Different cultural groups come about it. Good friend of mine, Roy Behat, you know, he runs Bloomberg Beta. He just tweeted yesterday that he says, when he shows DALL E to forget the term he uses, but I've gotta call it lay people. The people who are not steeped in technology, they are less impressed. They say, okay, so so because we work at this, we know how, you know, what it means, then we're impressed. So so there's that cultural difference. And then the other way, which is another friend of mine said, you know, who who would have thought that the first problems to be solved sort of credibly by machine learning, are the problems of creativity.
Right? We thought that was the final frontier. But now, we have models that are generating images based on, you know, just the description and so on, and beautiful paintings. So, yeah. It just reminds us to be very skeptical of what we think we know. Absolutely.
[00:46:41] Unknown:
And so for people who are looking for ways to be able to address a sparsity of data or challenges in being able to bootstrap machine learning capabilities? What are the cases where itematic and a human and knowledge driven first approach is the wrong choice?
[00:46:58] Unknown:
I would say if you can augment it with data. Right? Data augmentation comes in many different flavors. You can simply synthesize it. Right? You can, for example, in image processing, you can, you know, cut up the image, rotate it, scale it, and so on. And if that's sufficient, then you don't need to go directly to encoding human knowledge using our tool. It's not overkill, but maybe an inappropriate tool. But if you try that, and it's still not enough, because you know the information that you need, the knowledge that you need is not in that data. If you work it out logically, no matter what you do with the data, no matter how you extend it, is that gonna do it? Then I think the layer where we directly model human knowledge, I think that's where you need to operate. That's where Itomatic really shines.
[00:47:46] Unknown:
As you continue to build out the Itomatic platform in conjunction, the hfirst project. What are some of the things you have planned for the near to medium term or any particular projects or problem problem areas that you're excited to dig into? I've mentioned a few to you already. Right? Let's see what else can I mention?
[00:48:04] Unknown:
There's something that we do care a lot about, which is automotive. We're involved in project both in the in cabin experience of automotive, but also at the data layer, the the cybersecurity layer. Right? And for their own reasons, they impact human life. It's pretty fun to work in this industry where you know that it touches you in some way, shape, or form. So, for example, I don't know if you drive, Tesla. Our team has worked on the battery manufacturing science for for all of Tesla's batteries. Right? Sticking to sort of the automotive space, the in cabin experience, if you think about it, can be transformed by machine learning AI. That's a given statement. Right? But again, this is where, how do you build something using data where the equipment is not out there yet? So this is also where human knowledge, right, a good designer saying, you know, if the temperature is too warm, you know, the person has been sleeping at home, whatever, what combination of things that otherwise is just really just human creation, human creativity, and so on. If that can somehow be encoded using our system, our tool, to make your life better next year when you drive a car that's informed by these systems. I think some of those things, our team is very excited to work on. That's another interesting pattern where, because these systems have not been deployed, there's no data to collect.
And leveraging human expertise, human design in this case, and automating that somehow is also an area that we're looking into.
[00:49:40] Unknown:
So for anybody who wants to get in touch with you and follow along with the work that you're doing, I'll have you add your preferred contact information to the show notes. And as the final question, I'd like to get your perspective on what you see as being the biggest barrier to adoption for machine learning today.
[00:49:54] Unknown:
Biggest barrier for machine learning adoption. Certainly, I think well, we talked about data. Right? And data in the sense, you know, there's this sophistication where it's, okay, well, I I need to go collect data. But the next layer is what we talked about. You know? What if the information, the knowledge that you need is not in the data no matter how hard you try? Right? And this is more true for a lot of the physical companies. So you look at across the landscape. I have not seen a recent chart, but I bet you if you do a histogram of adoption of machine learning, right, you will find that it is highest in these digital first use cases and digital first companies, and lowest where you go more towards physics and then physical systems, and, you know, human life and limb, life critical systems, and so on. So I think that's if you could know that a barrier if you want, But, you know, having been part of Panasonic for 5 years, I kind of I went native. I became sympathetic to their causes. Right?
You know, sometimes sitting outside, we say, oh, you guys are too slow. You don't get it. People really get it. There are a lot of very smart people, but they're working on problems that are harder than problems that we're used to work on. Right? Because they make a mistake, somebody died, so they they have to be very careful. Absolutely.
[00:51:08] Unknown:
Well, thank you very much for taking the time today to join me and share the work that you're doing at i2matic and just helping to drive forward this question of human first AI. It's definitely a very interesting and exciting problem domain. So I'm happy to see the work that you and your team are doing. So I appreciate your time and effort on that, and I hope you enjoy the rest of your day. And thanks for the great question, the discussion.
[00:51:32] Unknown:
Thank you for listening. And don't forget to check out our other shows, the Data Engineering podcast, which covers the latest in modern data management, and podcast dot in it, which covers the Python language, its community, and the innovative ways it is being used. You can visit the site at the machine learning podcast.com to subscribe to the show, sign up for the mailing list, and read the show notes. And if you've learned something or tried out a project from the show, then tell us about it. Email hosts at themachinelearningpodcast.com with your story. To help other people find the show, please leave a review on Apple Podcasts and tell your friends and coworkers.
Hello, and welcome to The Machine Learning Podcast. The podcast about going from idea to delivery with machine learning. Predabase is a low code ML platform without low code limits. Built on top of their open source foundations of Ludwig and Horovod, their platform allows you to train state of the art ML and deep learning models on your datasets at scale. The prediabase platform works on text, images, tabular, audio, and multimodal data using their novel compositional model architecture. They allow users to operationalize models on top of the modern data stack through REST and PQL, an extension of SQL that puts predictive power in the hands of data practitioners.
Go to the machine learning podcast.com/predibase today to learn more. That's predibase.
[00:00:57] Unknown:
Your host is Tobias Massey. And today, I'm interviewing Christopher Ngoyan about how to address the cold start problem for ML and AI projects and the work that he's doing at Aitomatic to help address that as well. So, Christopher, can you start by introducing yourself?
[00:01:11] Unknown:
Hey. Thanks, Thomas. I am currently CEO and and cofounder of a company called Itomatic. So that's just like automatic except it starts with AI. Our specialty, our software, help translate human knowledge, domain knowledge into machine learning models. It's quite interesting. Right? Specifically for industrial companies where that the problem of not being able to take advantage of human knowledge is the greatest.
[00:01:35] Unknown:
And do you remember how you first got started working in machine learning?
[00:01:39] Unknown:
No. I've had a very long career. I started in Silicon Valley back in the early eighties as a child. I was a hacker, you know, even the days before the Atari 800, if you can believe it. So I've I've worked the entire stack from transistors all the way up to machine learning algorithms and so on. But I think the first application of machine learning statistical analysis, and so on was when I ran statistical arbitrage trading company in Asia. I spent 13 years back in Asia between, you know, the late nineties and and the early 2000. Back then, compute and data was not as plentiful as it is today, but certainly applying data to win on Wall Street. Right?
The d e shawls of the world, and I took the ideas there and and applied to Asian markets and, you know, created what's called a market neutral trading company. You know, if if you work on Wall Street, you understand these terms. But it's kind of typical that financial industry that and, you know, on the other extreme, the porn industry tend to be the ones that take advantage of technology the earliest because that's where the big the biggest profit margins are, I suppose.
[00:02:49] Unknown:
In the overall space of machine learning, there is this question of the cold start problem or the small data problem. And I'm wondering if you can kind of describe a bit about what that is and how it manifests and some of the ways that it impacts the ability of an organization to even be able to start thinking about investing in machine learning or what their machine learning capabilities might be.
[00:03:12] Unknown:
Yeah. So the cold start problem can be thought of as and anybody who is trying to apply machine learning algorithms into their actual, you know, work, into the actual applications, will run into this issue at some scale or other. Right? And machine learning, as you know, the food for machine learning is data. And so recently, there's been a recognition that data is not plentiful. Data is not as available as we wish it to be. And I think that's generally known and accepted now. I think what's much less appreciated is that there's a very distinct scale of that problem.
Meaning, everybody run this through data issues. But for a large class of problems, the solution is, say, well, go collect more data. You know, wait a a week or 2, wait a month or 2. But also for this $25, 000, 000, 000, 000 industrial economy, that answer is not good enough. It doesn't work. So I think generally, the people in the field understand that problem, but I think there's still not quite an appreciation for the size and scope and distribution of that challenge as applied to different segments of the economy.
[00:04:22] Unknown:
And then as far as the specific kind of application or architecture of the AI approach that you're taking, what are some of the ways that that can impact the scale or quality of data that you need in order to be able to even get started, where maybe if you have a certain data set, you can use that to bootstrap an ML model. And then once you have that in operation, it gives you a path to be able to feed new data sets in to kind of get the flywheel moving so you can continue investing.
[00:04:51] Unknown:
Yeah. I'll give you an example at 2 extremes that I have worked in. 1 is Google and the other is Panasonic. So when I was at Google in the early 2000, I was the first engine director for Gmail and Calendar and so on, but became Google Apps. And broadly speaking, data is very plentiful, right, for a company like Google. If you don't have the data that you need, you write 10 lines of code, you conduct what's called a percentage experiment, you launch it out to some subpopulation of your user in the morning, and by afternoon, you've got a 1, 000, 000, 000 examples. Right? There's no surprise that companies like Google, Facebook, Twitter, and so on. Is what I call digital first companies are also the first to be able to take advantage of the powers of machine learning. And then at the other extreme, like Panasonic, which acquired my last company, back in 2016, 2017, we basically went in there, you know, very gung ho trying to apply our AI machine learning and and hit the wall very quickly. And the wall could not be easily removed by writing 10 lines of code and launching it out to production, because things there move at physical speed. Right? And manufacturing, line, robot arm on that line, or fishing industry, and so on automotive, these things, you can't just go and conduct these experiments and say, Okay, I just want to, I need a 10, 000 example of car crashes. It doesn't exist. Right? You can try to simulate it, but it still moves at a certain speed.
So there began, you know, I sat down and I thought about it. There's a fundamental pattern here, sort of what I call digital first versus physical first types of companies. And so industrial companies, again, which still powers $25, 000, 000, 000, 000 annual economy, the techniques and the systems and the approaches that we have from Silicon Valley don't readily work. And so there needs to be a different approach, an augmented approach. And as you mentioned, you know, 1 of the approaches to do data augmentation at the data layer. Right? There are companies like Scale dotai and Snorkel and so on that addressing it at that layer.
A larger view is that we need to inject human knowledge, human domain expertise somehow into into that equation that the data sets that are coming off of the sensors on these systems are not enough. And so at the data layer, you can do that. But at the algorithm layer, there's much less awareness that is happening as well. And then that what I call the even higher layer, I call the something layer, where we combine human models, human knowledge models together with ML models seamlessly. That's also happening as well. Many companies like ours, you know, in in the context of a Panasonic or for Bruno or Tesla and so on, are applying it quite successfully. And, again, not just at the data layer. I'm also interested in understanding kind of what the
[00:07:43] Unknown:
scale of information is that you need if you're doing, you know, a deep learning model that is typically very data hungry versus a maybe, statistical or a stochastic model that you can, you know, be a little bit more granular in terms of how much information you want to use and how you want to kind of scale the model and some of the ways that you're able to maybe bootstrap that process by starting with a simpler model and then kind of ratcheting up to a more complex or sophisticated model architecture?
[00:08:13] Unknown:
The real wisdom is the right tool for the right job. Right? But I'll take you to an extreme, right, to see where you cannot have, you know, 1 tool for all jobs. Take the problem of predictive maintenance, right, in a physical industry, like predictive maintenance of refrigeration equipment, right, or predictive maintenance of aircraft engines. Invariably, in our industry today, you know, and you talk to a lot of people, and you say, hey, you guys doing predictive maintenance? Anybody who says yes will invariably say what they're doing is anomaly detection. Right?
And turns out anomaly detection is not a complete solution to predictive maintenance. All anomaly detection does, it takes in large amounts of sensor data, and then it can then you know, after 3 months worth of data or so, right, you can say, okay. Now I'm seeing it. Today's example does not look like what I've been seeing the last 3 months or the last 3 years, but it does not diagnose what's wrong. Right? In order to diagnose what's wrong, if you wanna take that to sort of the pure machine learning way, you need lots of examples of failures. Right? But by definition, you don't have a lot of examples of failure, particularly once you stratify it, model year, you know, by workload, you know, by operating conditions and so on. Really, there's, you know, whether it's deep learning or, you know, more statistical approaches or even small data machine learning, if you will. Machine learning itself, by itself, is not enough to solve those those problems. And what you really need to do is that okay. But a human with 30 years of experience looking at the situation, look at the sensor data, and so on, Not just from that dataset, but also from their years of experience can diagnose that and say, in about 2 weeks, I think this compressor is going to fail. So there needs to be an approach where you actively incorporate that human knowledge, not just at the data level, certainly at the data level, but also at the algorithmic level. And then the question becomes, did you go back to the old expert systems? Right? But that seems like a a step backward. The real thing is, are there opportunities to combine that seamlessly with this new machine learning world? And that's what companies like ours, IcoMatic, and a number of others are advocating.
[00:10:24] Unknown:
And so in that question of being able to incorporate the human in the loop and have their knowledge and experience and expertise feedback into kind of the model development and model design process, I'm wondering how you're able to kind of bridge the gap between the skill sets that are necessary to be able to design and build the model and the person who is maybe an expert in, you know, maintaining large diesel engines or aircraft engines and managing kind of that communication path because, you know, if you're a machine learning engineer working at your laptop all day versus a, you know, aircraft mechanic, you're probably not gonna be working in the same physical spaces. And so just being able to figure out how do you manage that kind of communication path and the collaboration flow across those kind of disparate roles and responsibilities.
[00:11:15] Unknown:
At minimum, 2 things are needed to get into this future. Right? As people say, the future has arrived, it's just unevenly distributed. Right? So look what companies like ours have done in the case of Panasonic and Furuno and other companies. So 2 things I need. Number 1 is that we we can't go back to the slow, you know, workflow heavy human in the loop. That's sort of pure and actual human sitting there in the loop. Right? We have to automate that. Right? There has to be a way to operate at machine speed, but with human intelligence. So that's sort of challenge number 1. But related to that challenge number 2 is as as you build a tool to do that, that tool has to be better than the alternative.
And the alternative is, you know, expect for sitting down and telling you or me to say, well, this is the rule. And then we encode that using, I thought, or whatever into the it has to be better than that. Right? So 1 approach that we are taking at Itomatic is interestingly taking advantage of the latest machine learning technologies. Right? You know, you have the emergence of these large language models. Right? Which and I'm gonna use an analogy. There's a lot of people disagree, but that's a different debate. These models seem to have a knowledge of the world. Right? So instead of having to show a million examples of, say, a circle. Right, you can just refer to the concept circle. So in other words, once you're able to communicate with these models, using natural language, then domain expert can sit down and say something like or type something like, if the pressure has been rising in the last 2 weeks, and the temperature remains constant, then you should take a look at the compressor.
Right? A natural language, literally, a phrase like that somehow gets easily converted into code that gets easily converted directly into a model that can then be productionized. So that's the challenge of, you know, a company like ours saying, okay, we're gonna translate human knowledge into ML models for you guys. In terms of that kind of progression of
[00:13:16] Unknown:
the advent of some of these kind of frontier models and large language models and kind of foundational models with transformer architectures and then being able to parlay that into a system where you can use some of those natural language exchanges to then generate subsequent models. I'm wondering how you see that kind of progression in the machine learning space of enabling that and where it is, you know, trending towards in the near future, and maybe, you know, contrasting that with the approach of kind of traditional software development where we are kind of building on the shoulders of giants with progressive layers of abstraction and maybe some of the potential pitfalls or risks that you see in the ML space as you kind of chart that same trajectory?
[00:14:02] Unknown:
I love how you think about that because the answer is embedded in the question. Yeah. You may not I think you asked it that way precisely because it is moving in that direction. I'll talk about the long term first and then say, you know, come back to the short term. What are we gonna do there? But what's implied, or at least I infer from what you're saying is, I'm gonna put it this way. Machine learning based purely on data is not the future. Right? Once you have these models, you should be building on the shoulder to the giants. Right? Machines will increasingly understand more than just raw data. So there's a near future where the training and, you know, people refer to it, you know, as prompt engineering, but I think it's much deeper and much broader than that. Right? We will be able to train machine models, right, with knowledge, because they already possess some knowledge. And then we sort of just ladder up. So the future of machine learning AI is really increasingly, let's use the word intelligent, increasingly intelligent machines or systems that can understand, right, 1st, raw bits and byte, pixels, and so on. And then next lower layer concepts, like circles and squares and so on, and maybe even a higher concept.
So then their ability to reach, you know, and converse with us. Right? Whether it's through audio or to type language. Their ability to take direction and get trained will be increasingly more powerful. Right? That's absolutely the future. So that's why when sometimes when I talk about these people say, you know, you talk about expert systems. Well, I say, well, you can live by expert system, but on steroids. Right? Because these machines themselves understand what you're telling them. In the near term though, you know, before we reach that laddering, then there needs to be the initial translation layer. Right? So what we're doing at Itomatic is really the large language models are being used as that API to do the translation, and translate it into a DSL. Right? And it's strictly typed and syntax and so on. And then from there, we go into an architecture that has a teacher, a student, and an ensemble. The teacher is the model that was generated from that translator.
The student is a standard ML, you know, data driven model, but the teacher is sitting there training the student, you know, with various, you know, system data. And then when you assemble them together, it turns out that human model and the machine learning model can combine to be the best of 2 worlds.
[00:16:24] Unknown:
Continuing on this kind of topic of being able to progressively build more sophisticated machine learning models because of the machine learning capabilities that already exist and being able to use some of those kind of foundational models to help generate some of the subsequent stages, you know, drawing that parallel back to software development and the software supply chain. There are a number of issues that have come up in more recent years because of the kind of level of scale and sophistication and adoption of the software systems that we're building in terms of things like supply chain security, you know, managing the kind of logical complexity of systems as they scale out, and, you know, understanding how to manage the kind of continuous integration, continuous deployment.
I know that there are a lot of those same problems that are being tackled in the machine learning space as well, but I'm wondering what you see kind of as some of the up and coming challenges that the machine learning ecosystem is going to have to tackle drawing on lessons from the past?
[00:17:28] Unknown:
Absolutely. In fact, that reminds me of a symposium that is coming up in November called Knowledge First World. And it's quite distinct from other machine learning AI conferences in the sense that it is not a research conference. It is a practice conference. Leaders from government, there's a major general from the NSA coming, right, from academia. Jeanette Wing, former head of Microsoft Research and the current EVP of Research at Columbia and of Industry, you know, Panasonic folks, leaders, they're basically gonna go then compare notes. And the notes that they're comparing certainly include, you know, successes and failures in the application of these technologies.
But because these are, you know, life critical systems and processes, a lot of us in Silicon Valley and the digital economies, we have not yet directly dealt with. So for example, automotive technology, avionics, and so on. The ethical, the safety issues are ever present in their mind, even from day 1 before they build the first system. It's not it's not a, you know, afterthought. So a lot of those issues occur in those forum. The difference is that even even at present, these tend to be separate discussions, right? You go to ICML, and you talk about algorithms, and then you go to an event organized by NIST, and they talk about trust and human safety, and the expertise do not overlap. So I think there needs to be this conversation like this knowledge first world symposium where the 2 topics are discussed pretty much by the same people informed, you know, by the same systems.
I think there'll there'll be a lot more of that driven by this industrial economy far more so than the digital economy because when you make a mistake with, you know, a a digital app, maybe the customer, the user clicks on the wrong ad. But when you make a mistake with with a self driving system, then somebody dies.
[00:19:22] Unknown:
To that point of kind of coalescing different disciplines and coalescing different concerns, as you are incorporating humans in the loop of helping to generate the models that are going to power your operations, power your applications and business. Humans aren't inherently limited to a single kind of mode where you're not necessarily thinking in terms of, oh, I'm only going to be working in natural language, or, oh, I'm only going to be working in computer vision, or, you know, I'm only going to be working on time series. As humans, we experience all of those things simultaneously, so I'm curious how that question of kind of multi model development and model training factors into the ways that you're thinking about human first AI development and the need to be able to
[00:20:12] Unknown:
holistic operating whole? That is essentially the gap between the present and the future that I alluded to. Right? So at present, as a tools company, right, we rely on what's available out there. We're not here to build a large language model, we're to use a large language model to actually do something, you know, very useful and powerful for the industrial companies. So the mode that is available to us today is that natural language mode. But you're absolutely right. There's already people saying, you know, can I just talk to it? Can I sit there and, you know, record something, and then you take care of it, Of course, then you can just do a transcript and so on? But vision. Right? How do you take advantage of what a fisherman sees? There's a use case we work on where, you know, you think about how a fish gets from the ocean to your table. There's a company called Furuno, which is a global marine navigation giant, and they have what's called a fish finder, both for the enthusiasts, possibly like you and myself, all the way to large fishing vessels.
But imagine the ultimate future when you shoot down the sonar beam, and it comes back pictures of schools of macro and so on. Today, that's not the case. Today, it comes back as an echogram. And it turns out, even the people who build these systems are not experts at interpreting those echograms. You think of it as ultrasound, but much messier. And it turns out these fishermen, in this case in Japan, and there's like a 150, 000 of them, and each 1 is expert diagrams, and they said, well, okay. Well, that's clearly a bunch of sardine, and and I'm looking at it. I'm saying, that's just, like, red with blue. I don't know what that is. Right? So capturing what they see and then translating that to, essentially, this video that you and I would see as a fish is an enormous challenge. And that's something that we're working with Furuno to do exactly that. So that vision. Right? But eventually, you know, taking advantage of human knowledge and human expertise, the experience that is inside the expert's brain, after all, came through through these modes. Right? Through what they hear, through what they feel. 1 more example as I talk about it. On a manufacturing line, as we work on predictive maintenance for these equipment, I hear of a line manager, every morning, he would walk down the equipment line, and he would place his hand, right, on the machine that's just being turned on, and he would say, take this 1 down for maintenance, and the other 1, no, that's okay. And so there's something that he's feeling from the vibration or whatever of the machine as he walks by. That should be distilled and captured and scaled because there are not too many of these experts around anymore.
[00:22:49] Unknown:
And that also brings in the interesting question of the modality of touch, which is something that is, you know, still very much in kind of the early days of being able to say, like, how can we teach machines how to actually experience that sensation of touch and texture and feel versus just measuring vibrations because you happen to have a sensor that, you know, is looking at the vibrational frequencies.
[00:23:11] Unknown:
I know exactly what you mean. We have 1 project is to, you know, detect it. Again, predictive maintenance, this time of robot arms on the manufacturing line. And we can't go there and disturb, you know, the thing that's happening. So we use, believe it or not, EM signature. Right? Electromagnetic waves. So you use this loop, and you sort of put it on the arm. And as it's operating, you get an EM signature off of it. And that is 1 mode. Turns out that is not something that humans can perceive. But on the other hand, the human touching that robot arm will feel the vibration in certain ways. And those are 2 different data sources. Right? To machine learning algorithm, they don't look anything like each other. But if they can be fused and combined somehow in in a multimodal or architecture, then I think we can, you know, emulate or copy from that human expert.
[00:23:59] Unknown:
And so digging into the platform that you're building, Itomatic, and the kind of goals of how you're looking to be able to attain this human integrated approach to machine learning and model development and bootstrapping machine learning models because of a sparsity of data. I'm wondering if you can just talk to some of the kind of infrastructure and design understandable to your target users.
[00:24:29] Unknown:
The key, as a matter of product design, is not to go in and say, I'm gonna revolutionize what you do. I said, no. No, please. Right? This thing is working. Can you just make it better? So the product goal is to produce what is essentially a machine learning model. Right? And then whatever system that you're using, whether it's MLflow, Kubernetes, and so on, it gets deployed the same way. But how it is the provenance of that is extremely interesting. Right? It is no longer only data streaming in and then running through, you know, TensorFlow or PyTorch and so on. But there's suddenly a new path, and that path starts with a human expert sitting down, perhaps typing in directly. Right? And at this point, the product articulation is that natural language, or working with an engineer, and the engineer would type this in. And so that pipeline is the only thing that is new and different. Right? It's a very powerful pipeline. It's able to take human input, and then outputs a model that works seamlessly with the entire system. That's our new term product rule. But, you know, the vision for the whole company is very much along the lines of you what you alluded to, which is, you know, how else can we take advantage of of human knowledge before it's gone?
[00:25:44] Unknown:
As you have been exploring the space of figuring out how to integrate the human experience and kind of human understanding of these industrial systems and translate that into a model training and model development process. What are some of the ways that your kind of initial design and goals and your understanding of the scope of the problem have changed as you have kind of gone from inception to where you are today?
[00:26:11] Unknown:
1 of the key things in that is that every use case is different. Right? So the idea is very broad. We actually have an open source project called human first AI, where we put a lot of these, you know, concepts and code out there. But in terms of applying it to industry, the ones that are most, number 1, sort of receptive and most resident at the moment, as I mentioned, are not the digital first companies. It's more the physical first company, the refrigeration, the automotive, the oil and gas, right, the fish finding industries and so on. And each of those has a different in specifics.
But the amazing thing is, as I talk to these folks, I don't have to sell them about human expertise. They actually sell it back to me. They said, we've been wanting to do this for a long time. We just don't have the tool to do so. Right? When we've been trying to do what is the the normative way of machine learning and collect data, and we find that it's not just that it's expensive to collect, it's also very long. And in many cases, like a machine learning prediction model. Right? You don't have examples. No matter how long you collect, You can collect that and suddenly the next model year comes in and you start from scratch again. So but coming back to your question, each case, each use case, you know, has to be sort of thought through very carefully.
And that's why we are more of a tools company. We're not a consulting company. We have gained some domain expertise simply by working with the customers. But what we're really doing is we're enabling the AI engineers in these companies to work on these things. So the customers that we work with tend to be actually quite far along the AI adoption curve. They're not people who say, what can I do with machine learning? They have tried and failed. Right? And they say, I need some tool to fill this gap. On that question of
[00:27:59] Unknown:
even just understanding when machine learning is a useful approach or a useful tool, You mentioned that the companies you're working with have already established that fact, and they've moved on to, okay, this first try didn't work. How do we do it better? But as you do start to kind of grow your operations and you start to onboard new customers, what are some of the ways that you help them to understand that initial question of, is machine learning a useful application for my problem, and how do I start to think about framing this problem in a way that machine learning is able to help facilitate the solution?
[00:28:37] Unknown:
To be sure, what we deliver is through machine learning. Right? But it is a more broadened view of where these models can come from. In some sense, this is what professor at academics have said for a very long time. In the last 10 years, suddenly deep learnings kind of took over, right, and took all the oxygen out of the room as it were. And then a bunch of, you know, old cranks say, hey, you don't need deep learning for that. You can just use, you know, SVMs or other things. And I think that voice is sometimes lost. Right? And so, you know, we take a step back and say that AI is broader than just machine learning, certainly much broader than just deep learning. Right? And so the conversation we have is just like, you know, is the right tool for the right job? Would you like this additional tool? Right? And for many use cases, you don't need this tool. But for a vast industry out there, this tool is vital. And it's still part of their entire machine learning pipeline. The deployment takes place, you know, in standard AWS. And if they already have a machine learning pipeline, this works seamlessly with it. But it gives them extra sort of superpowers.
The main epiphany is, even for us, is, like, these companies have vast domain expertise. Right? They've been around for 20, 30 years. In some way, Silicon Valley, including myself, sometime we tend to go in and we say, you guys step aside. Right? Let me connect this up. Let me collect the data and I'll do everything that's needed. It turns out that doesn't work. I mean, I don't mean just culturally, but actually technically. So having a tool like this is not just technically pleasing, but it's also culturally appropriate for a conversation with these, both the executive level and and the line management level at these companies.
[00:30:22] Unknown:
As far as the adoption process for the organizations that you're working with, can you talk to what's involved in actually onboarding onto itematic some of the existing capabilities that are either useful or necessary for them to have and some of the ways that you think about the kind of tool chains and platforms that you're aiming to integrate with, whether that's the, you know, specific ML library that they're focused on or the deployment methodologies or kind of training capabilities, things like that.
[00:30:52] Unknown:
So the typical team that we work with, as I mentioned, is that they they already have something. Right? They've been trying, and somehow it leaves them, you know, wanting something that's missing. And they feel, are they crazy? Am I wrong? You know, why are other companies succeeding and we're not? What are we doing wrong? So we go in and essentially, we say, well, you're not crazy. You really want to incorporate that human expertise. Right? Not just because it is the thing that makes it possible, but we should go to market, right, in 3 months, as opposed to trying to collect data, perhaps even unsuccessfully or augment data for the next few years. So in that context, they already have something, they have a pipeline, they understand TensorFlow, they have Python programmers, they data science teams, and so on, PyTorch, whatever it is that they're using, the output of this tool that we have for them, this translator, is a machine learning model. Right? It happens to be an ensemble of a teacher and a student. And the teacher happens to be knowledge based, and the student is machine learning. But as far as the API is concerned, it's features in prediction out. That plays very nicely with whatever production system they already have. And, of course, as part of a whole product suite, we also have what's the what I just described is the what we call the knowledge first build, k first build to to build models.
A lot of times because of that, they also wanna deploy with us. So we have something called k First Execute, which helps with deployment management and so on. And And even, you know, looking at the operation, you know, out in the field, and then looking back and sort of improving it with additional data as it sits out there for 6 months and the model can be improved.
[00:32:35] Unknown:
You mentioned earlier that you have this HFIRST or HumanFirst open source project that is intended to encapsulate some of the same ideas and principles as what you're driving at with Itomatic. And I'm curious if you can talk to the relationship between what you're building at Itomatic and what the h first library exposes. Like, is it is h first a kind of core component to what you're building at Itomatic, or are they just kind of mirrors of each other where you're, you know, releasing some of the lessons learned from Itomatic into the h first project and they're not necessarily
[00:33:06] Unknown:
a direct dependency of yours? That's a great question. No 1 just asked me that. I can see a diagram that I should draw. So I talk a lot about, you know, this pipeline that starts with a human, you know, whether it's typing in Japanese or English, and so on. And then pops on the other side is a teacher model. Right? And then the teacher model by itself doesn't do anything yet. So it has to go into some target architectures. Right? So these architectures are architectures that anticipate the addition of human knowledge. Right? We talk about that the data layer, there's data augmentation, you know, companies like, you know, scale dotai and Snorkel and so on, that are doing documentation. At the modeling layer, there has to be some architectures that somehow combine this human model with the machine learning model, ensembles, and so on. Right? So to answer your question, all of those architectures that we're creating and innovating on and other companies, other teams as well. Right? If you look at K First World, the the symposium that I mentioned, Google Cloud AI is coming to talk about how they are architecting a human knowledge pipeline together with the data pipeline coming out with a with a back prop that essentially train the whole model. So lots of these interesting architectures.
So the human first open source project is a collection of all of these architectures. Right? And then also some of the use cases as examples of how they're being used. The translator, that is something that Itomatics is building. And other companies can build it too. But that that's not part of the of the human first project. So the the you can think of it as dividing it Human to model, that's Itomatics product. And then once you have this human expert model, what architecture does it go into? That's the open source project.
[00:34:54] Unknown:
As you have been building the kind of commercial entity and the focus there and iterating on the open source capabilities that you've embedded into the h first utility. I'm curious if you can talk to some of the lessons that you've learned at the boundaries between them and some of the ways that you have kind of factored some of the community knowledge and community experience into how you think about problem that you're solving at IDOMATIC?
[00:35:20] Unknown:
Your question can be thought of different ways. For example, you know, you can think of it as, you know, what is an open source project, and and how do different companies think about open source and and and so on. And then the other is, you know, how do you commercialize something that is open source? Maybe I'll think about your question that way and talk through it. The second part, because I've been in this industry, I've been in tech for for a very long time. I worked on the first flash devices that that is now all of our bodies and so on. At the time, we thought it was only automotive applications. There was a time when commercial open source was an oxymoron.
Right? CIOs say, I work with Microsoft. I work with Google. Right? But I'm not gonna rely on this thing that is sort of unfunded and so on. Today, that decision has completely flipped 180 degrees. People prefer to buy, people prefer to use open source. You know, if something happens, they're not encaptured by a single vendor. Even if the vendor that the company that is supporting that project, even that, you know, they go belly up, the code is still there. So in principle, they can still run their own thing. And the quality open source, of course, and I I don't mean across the board, but certainly some of the most popular project, the the quality is is pretty clear.
From a commercial point of view, I'll mention a good friend of mine, Joseph Jacks, runs this fund. This is entirely dedicated to commercial open source. In fact, it's called OSS dot Capital. The entire portfolio is an open source, commercial open source companies. So I think the economic model is well proven. In terms of how CEOs or founders think about open source, there's still a very big range. I'll tell you the wrong ways to think about open source. Right? 1 wrong way about open source is to think that is the funnel. It's a marketing funnel. Right? Oh, I'll just put this thing out there, and then a lot of people will use it, and then some of them will convert. The lesson of the last 10 years is that open source users come in 2 flavors, and they don't automatically transform 1 into the other. Right? The free people will always be free. Right? The commercial people are commercial for an entirely different reason. They choose open source, not because it's cheap, they choose open source, because it is, you know, available and and and, you know, they're not locked in, but they still want the support and the development from a commercial entity.
So for the founder, you know, it's it's I'm gonna write some code. I'm gonna put it out there, and, you know, I'll just start collecting money. Good luck. Right? So in other words, a company like ours, even though we have this open source component, and we truly believe in it, we put it out there for contributions and so on, It's still our responsibility to invest in marketing and sales team and consulting and so on to to go and win these accounts 1 by 1. Right? Because those people are not sitting there and saying, let me download. The open source availability does add to their confidence, but it is not the main reason why, you know, they choose to make the $1, 000, 000, 000 decision on this tool. Absolutely.
[00:38:22] Unknown:
The whole question of open source and business is always interesting to get people's opinions on it because it is 1 of those polarizing debates, you know, in the same category of tabs versus spaces or Mac versus Windows or the age old wars that will never die.
[00:38:39] Unknown:
Right. Right. And is it tab 4? Is it tab 8?
[00:38:43] Unknown:
Or e max versus buy? Or
[00:38:46] Unknown:
That's been long settled. Definitely vi.
[00:38:49] Unknown:
I was just gonna say e max and 4 spaces.
[00:38:54] Unknown:
I'll let you get away with 4 spaces, though.
[00:38:59] Unknown:
And so in terms of your experience of building IDOMATIC and the h First project and working with your customers and helping them understand how to encode human knowledge and human experience into ML models that allow them to scale their business and increase their operational efficiencies? What are some of the most interesting or innovative or unexpected ways that you've seen IDOMATIC and HFIRST applied?
[00:39:23] Unknown:
Maybe I'll talk it from, you know, the innovation. A lot of it comes from our customers. Right? So when we first started doing this, we were thinking, and it's still the case, many of the use cases is, you know, you have 2, 3 experts, the coaching, the refrigeration, the project that we work for Japan, you know, Japan, they're convenient store, but they're actually very large scale markets like 711. 711 in the US at all. Right? 711 is a large supermarket in in Japan. Family Mart, Lawson, and so on. So they have, in every store, they have lots of refrigeration equipment.
And there are 3 experts in the entire country of Japan that are available if you want to do this predictive maintenance, that can diagnose in advance what's likely to fail. So we tend to build these systems with companies that are running out of experts who don't have enough and to scale them. Then then we hear from a customer, the fish find use case. Right? And they say, can you help scale this? So the knowledge problem is not 1 expert, you know, 1 model, or 3 experts, 1 model. Can you take in the knowledge of a 150, 000 fishermen?
Right? Of course, we can just, you know, throw it out there and say, you know, go ahead and type into here, and then we'll generate a model. Could ask for fundamental question, what does it mean to integrate the knowledge of a 100 50, 000 humans? I don't know the answer to that yet. That's still a question in life. Right? But it does drive a lot of innovation, a lot of thinking among our research team. Do I just deploy a 150, 000 models? Right? And say this model is good for Hokkaido, and this 1 is good for Okinawa. But that's a very limited view. Clearly, there is something in the knowledge of these people that transcend geography, transcend, you know, day or night, transcend weather, and so on. But what does it mean to combine those things, you know, other than just a simple straight ensemble of these things in a way that's sort of optimal, right, in a way that we can have, you you know, can we have the intelligence of a 150, 000 people?
Right? Making decisions day in, day out. If we succeed at that a few years from now, that'll be a major milestone of accomplishment for me, not just for the company. Right? But these these are the thing that make me jump out of bed every morning and say, hey, I wanna go to do that. Right? Because, you know, you have this idea, you have this tool, and then your customers say, you know, hey, can you help me do this? And then it sort of changes the game of how you look at it. Another thing that we didn't dig into yet is kind of what are the
[00:41:50] Unknown:
natural limitations of this approach? What are some of the ways that you have to constrain the possibility space in order to make it something that is tractable for converting into a machine learning model where you obviously can't just, you know, download the entire experience of an expert fisherman or a, you know, expert mechanic at some industrial firm. And so just understanding how to kind of define and understand what are the appropriate bounded contexts in which you are trying to kind of build a target model and some of the ways that you are able to kind of understand the current limitations
[00:42:29] Unknown:
and start driving towards pushing those further so that you can kind of expand the range of capabilities and modalities that you're able to operate in. That is the essential difference between the next 3 months and the next 30 years. Right? I mean, a lot of us on Silicon Valley, the mistake we made is not that we're too late, that we're too early. Right? We tend to build things that make a lot of sense 10 years from now. But people buy things that make money for them 3 to 6 months from now. So you're absolutely right. What we do is, you know, 1 of the things that we want to be evaluated for, you know, is not that we, you know, human being come in, and then somehow we put probes on your head and then pull them a copy of you comes out. That's not what we're selling. What we're selling is we can do something that is much better than the baseline of your current approach.
Right? And typically, it's not 10% better. It has to be 10 x better. Right? It suddenly unlocks the ability to encode human knowledge, you know, that can do some of the heuristics. Right? It's certainly not better than human. It could be more consistent. It could be more scalable and so on. Right? But compare us against the baseline of not having it. And that, you know, when people take a look at that and say, oh, wow, I can go to market in 6 months, and I can slap this AI label, you know, on my system that actually does something useful. And I think that's, you know, that's enough reward for the next, you know, 2, 3 years to just to help company go to market and make a difference, a marginal, you know, an improvement on their product that otherwise would not be possible without this encoding of human knowledge.
[00:44:02] Unknown:
In your experience of working in this space and building your business, what are some of the most interesting or unexpected or challenging lessons that you've learned in the process?
[00:44:11] Unknown:
That it is harder to talk to Silicon Valley or say machine learning people about this problem than I expected. And conversely, it's a lot easier to talk to the people that are otherwise not machine learning experts. Somehow, we have this bias, and I'm partly responsible for it. So, you know, data will save us all. Right? Big data was was a big cry, just short 10 years ago. People are using this. And yes, it is in the in the future where you and I just talked about earlier, right, where data will sort of just ladder itself up. Right? Data will become knowledge, knowledge will become experience, hopefully, an experience become wisdom, and all of that is somehow automated. Right? But, you know, now it's where we see generally, but I I know that while I was still at Panasonic and talking to my friends, I said, hey, man. There's not enough data. They would say, well, just go collect more. I said, no. You is it not that simple?
That's 1 of the biggest surprise. It's no longer a surprise, but initially, I received the biggest resistance. Right? The admission that this is in fact a problem and that the problem worth working on, you know, from the very people that are the experts at these algorithms.
[00:45:20] Unknown:
Yeah. It's definitely funny as we reach what in different problem domains, what we think are kind of revolutionary capabilities or revolutionary approaches to solving a problem because of the fact that we're bounded by our view as engineers or, you know, programmers, data scientists. And then we talk to people who we're trying to solve the problems for, and they say, oh, I thought it already worked that way.
[00:45:44] Unknown:
Yeah. That's right. Yeah. Different cultural groups come about it. Good friend of mine, Roy Behat, you know, he runs Bloomberg Beta. He just tweeted yesterday that he says, when he shows DALL E to forget the term he uses, but I've gotta call it lay people. The people who are not steeped in technology, they are less impressed. They say, okay, so so because we work at this, we know how, you know, what it means, then we're impressed. So so there's that cultural difference. And then the other way, which is another friend of mine said, you know, who who would have thought that the first problems to be solved sort of credibly by machine learning, are the problems of creativity.
Right? We thought that was the final frontier. But now, we have models that are generating images based on, you know, just the description and so on, and beautiful paintings. So, yeah. It just reminds us to be very skeptical of what we think we know. Absolutely.
[00:46:41] Unknown:
And so for people who are looking for ways to be able to address a sparsity of data or challenges in being able to bootstrap machine learning capabilities? What are the cases where itematic and a human and knowledge driven first approach is the wrong choice?
[00:46:58] Unknown:
I would say if you can augment it with data. Right? Data augmentation comes in many different flavors. You can simply synthesize it. Right? You can, for example, in image processing, you can, you know, cut up the image, rotate it, scale it, and so on. And if that's sufficient, then you don't need to go directly to encoding human knowledge using our tool. It's not overkill, but maybe an inappropriate tool. But if you try that, and it's still not enough, because you know the information that you need, the knowledge that you need is not in that data. If you work it out logically, no matter what you do with the data, no matter how you extend it, is that gonna do it? Then I think the layer where we directly model human knowledge, I think that's where you need to operate. That's where Itomatic really shines.
[00:47:46] Unknown:
As you continue to build out the Itomatic platform in conjunction, the hfirst project. What are some of the things you have planned for the near to medium term or any particular projects or problem problem areas that you're excited to dig into? I've mentioned a few to you already. Right? Let's see what else can I mention?
[00:48:04] Unknown:
There's something that we do care a lot about, which is automotive. We're involved in project both in the in cabin experience of automotive, but also at the data layer, the the cybersecurity layer. Right? And for their own reasons, they impact human life. It's pretty fun to work in this industry where you know that it touches you in some way, shape, or form. So, for example, I don't know if you drive, Tesla. Our team has worked on the battery manufacturing science for for all of Tesla's batteries. Right? Sticking to sort of the automotive space, the in cabin experience, if you think about it, can be transformed by machine learning AI. That's a given statement. Right? But again, this is where, how do you build something using data where the equipment is not out there yet? So this is also where human knowledge, right, a good designer saying, you know, if the temperature is too warm, you know, the person has been sleeping at home, whatever, what combination of things that otherwise is just really just human creation, human creativity, and so on. If that can somehow be encoded using our system, our tool, to make your life better next year when you drive a car that's informed by these systems. I think some of those things, our team is very excited to work on. That's another interesting pattern where, because these systems have not been deployed, there's no data to collect.
And leveraging human expertise, human design in this case, and automating that somehow is also an area that we're looking into.
[00:49:40] Unknown:
So for anybody who wants to get in touch with you and follow along with the work that you're doing, I'll have you add your preferred contact information to the show notes. And as the final question, I'd like to get your perspective on what you see as being the biggest barrier to adoption for machine learning today.
[00:49:54] Unknown:
Biggest barrier for machine learning adoption. Certainly, I think well, we talked about data. Right? And data in the sense, you know, there's this sophistication where it's, okay, well, I I need to go collect data. But the next layer is what we talked about. You know? What if the information, the knowledge that you need is not in the data no matter how hard you try? Right? And this is more true for a lot of the physical companies. So you look at across the landscape. I have not seen a recent chart, but I bet you if you do a histogram of adoption of machine learning, right, you will find that it is highest in these digital first use cases and digital first companies, and lowest where you go more towards physics and then physical systems, and, you know, human life and limb, life critical systems, and so on. So I think that's if you could know that a barrier if you want, But, you know, having been part of Panasonic for 5 years, I kind of I went native. I became sympathetic to their causes. Right?
You know, sometimes sitting outside, we say, oh, you guys are too slow. You don't get it. People really get it. There are a lot of very smart people, but they're working on problems that are harder than problems that we're used to work on. Right? Because they make a mistake, somebody died, so they they have to be very careful. Absolutely.
[00:51:08] Unknown:
Well, thank you very much for taking the time today to join me and share the work that you're doing at i2matic and just helping to drive forward this question of human first AI. It's definitely a very interesting and exciting problem domain. So I'm happy to see the work that you and your team are doing. So I appreciate your time and effort on that, and I hope you enjoy the rest of your day. And thanks for the great question, the discussion.
[00:51:32] Unknown:
Thank you for listening. And don't forget to check out our other shows, the Data Engineering podcast, which covers the latest in modern data management, and podcast dot in it, which covers the Python language, its community, and the innovative ways it is being used. You can visit the site at the machine learning podcast.com to subscribe to the show, sign up for the mailing list, and read the show notes. And if you've learned something or tried out a project from the show, then tell us about it. Email hosts at themachinelearningpodcast.com with your story. To help other people find the show, please leave a review on Apple Podcasts and tell your friends and coworkers.
Introduction to The Machine Learning Podcast
Interview with Christopher Ngoyan: Addressing the Cold Start Problem
Understanding the Cold Start Problem in Machine Learning
AI Approaches and Data Challenges
Predictive Maintenance and Human Knowledge Integration
Future of Machine Learning: Building on Foundational Models
Challenges in Machine Learning Supply Chain
Multimodal Development and Human-First AI
Itomatic Platform: Integrating Human Knowledge into ML Models
Onboarding and Adoption of Itomatic
Open Source and Commercialization
Innovative Applications of Itomatic and HFIRST
Lessons Learned and Challenges in Human-First AI
Future Projects and Problem Areas
Biggest Barriers to ML Adoption
Conclusion and Closing Remarks
