Peculiar Inklings

This is the current home of the Loma Buena Associates Peculiar Inklings blog. What appears here are miscellaneous comments and observations about data science, analytics, AI, and "principled" methods, authored by not-so-prolific but generally approachable authors of some experience. 
Location: Salt Lake City, UT, USA

Popular posts from this blog

Are you Certain About How Uncertain You Should Be?

Image
This Blog Post: Are you Certain About How Uncertain You Should Be?  © 2024  by  Lynd Bacon & Loma Buena Associates  is licensed under   CC BY-SA 4.0  TL;DR?  Here's the BLUF: It has been said that love is a many-faceted thing.  So is uncertainty.  Because it is, the multiple sources of it in any data science or analytics effort should be well documented and expressed as precisely as possible.  Doing otherwise is not good practice, and is not in the best interest of supporting the best possible decision-making based on results. Life is chock full of uncertainty.  So are research, prediction, and data analytics. I've recently been considering some challenges in expressing and communicating uncertainty. I've also been thinking about the critical elements of reproducible data science workflows that I need to emphasize for my students and colleagues. I find myself once again reminded of how difficult it can be to understand uncertai...
Read more

Risk Controlled Conformal Prediction of Outcomes for Customers, Patients, or Other Entities

Image
This Blog Post: Risk Controlled Conformal Prediction of Outcomes for Customers, Patients, or Other Entities  © 2024  by  Lynd Bacon & Loma Buena Associates  is licensed under   CC BY-SA 4.0  TL; DR?  Here's the BLUF: Risk-controlled conformal prediction methods subsume conformal prediction methods as a special case.  They allow controlling for the risk of specified types of prediction errors.   Single or multiple kinds of risk can be accommodated.  Like conformal prediction methods, risk-controlling methods are model agnostic, and they can provide finite sample statistical certainty guarantees.  A simple, and mostly conceptual, description of risk controlled conformal prediction, follows. As noted elsewhere and by many, conformal prediction methods can provide a statistically valid, finite sample certainty guarantees about whether predictions for individual new cases, contain, or cover, the "true" quantity or value to be ...
Read more

Conformal Prediction: Simple Model-Agnostic Prediction with Statistically Guaranteed Error Rates

This Blog Post about Conformal Prediction © 2023 by Lynd Bacon and Loma Buena Associates, is licensed under CC BY-SA 4.0    TL;DR? Here’s the BLUF:  "Conformal prediction methods can provide you with statistically valid certainty guarantees regarding the likely values of predictions using “new” data, data that you haven’t observed outcomes for, or that you just didn’t use for training and validating models." A simple classification example with Python code snippets follows below, along with some suggested follow-up resources.    Suppose you’ve trained, tuned, and validated a predictive model that you will use to predict outcomes for new “objects,” new cases or observations. Your model might be simple regression or classification model, an ensemble model of some sort, or a deep neural network. Your objects might be customers and their responses or their types, drug molecules and their efficacy, or any other sort of thing that you use a model to p...
Read more