A practical approach to measuring and using the prediction uncertainty of a natural language processing system
Can a model with 90% accuracy be useful, if the task strictly requires 98%? The answer is, probably yes, but not always. Thanks to the latest research in uncertainty estimation for Natural Language Processing models, the model can not only predict its own score for each row, but also what is the chance a specific prediction will be correct. This chance is used to filter the most uncertain predictions, leaving only high-quality and accurate ones to AI, and sending those unclear for manual human review. During the talk, I’m going to present a practical approach to measuring and leveraging the prediction uncertainty on an example of commercial, production-grade deployment of an NLP model.
Marek is an expert in the field of natural language processing applications and predictive analysis for marketing and sales powered by artificial intelligence with 9 years of commercial experience. He has been passionate about AI since the age of 16, his scientific activity was honoured by professors of Harvard University and the Director of NASA in the form of an official recommendation letter when he was 19. At the same age he was invited by NASA to lead a project on optimisation of the trajectory of an international space station to increase the amount of electricity from solar panels. During his studies, he repeatedly traveled to Microsoft and NVIDIA in the USA for internships. He also became the best student of the year at the University of Warsaw in machine learning.
He started his career working at Microsoft headquarters and later for NVIDIA in Switzerland as a senior scientific engineer of artificial intelligence, working on autonomous cars. After a year of full-time work, he founded a consulting company that carried out projects for many corporations in the field of artificial intelligence implementation in marketing, sales and finance, including one of the 3 largest pharmaceutical companies in the world. His experience and projects have been covered many times in the international press, including the Financial Times and Bloomberg.
At 26, he generated multi-million dollar AI projects returns for his clients. He is currently developing his international AI consulting company and foundation that helps homeless youth in the Philippines with education with the ambitious goal of educating participants within 2 years as programmers and providing them with remote work, while addressing the problem of loneliness of people in developed countries at the same time.