Peculiar Inklings

This is the current home of the Loma Buena Associates Peculiar Inklings blog. What appears here are miscellaneous comments and observations about data science, analytics, AI, and "principled" methods, authored by not-so-prolific but generally approachable authors of some experience. 
Location: Salt Lake City, UT, USA

Popular posts from this blog

Conformal Prediction of Customer Segment Memberships by Customer Type; Uncertainty, Quantified and Unquantified

This Blog Post: Conformal Prediction of Customer Segment Memberships by Customer Type; Uncertainty, Quantified and Unquantified © 2024 by Lynd Bacon & Loma Buena Associates is licensed under CC BY-NC-SA 4.0 TL;DR? Here’s the BLUF: Conformal prediction methods can provide statistically valid certainty guarantees when predicting class (e.g., segment) memberships, or other kinds of outcomes, that machine learning models are often used for predicting. Depending on how these methods are used, the quality of the results can depend on characteristics of the objects (e.g., customers) that predictions are for. When such characteristics consist of a priori known types or groups, investigating the quality of conformal prediction results for them is very straightforward. Conformal prediction is for quantifying uncertainty. Some kinds of uncertainty that decision-makers face may not be quantifiable.   This conformal prediction niblet is a follow up to a pr...
Read more

Are you Certain About How Uncertain You Should Be?

Image
This Blog Post: Are you Certain About How Uncertain You Should Be?  © 2024  by  Lynd Bacon & Loma Buena Associates  is licensed under   CC BY-SA 4.0  TL;DR?  Here's the BLUF: It has been said that love is a many-faceted thing.  So is uncertainty.  Because it is, the multiple sources of it in any data science or analytics effort should be well documented and expressed as precisely as possible.  Doing otherwise is not good practice, and is not in the best interest of supporting the best possible decision-making based on results. Life is chock full of uncertainty.  So are research, prediction, and data analytics. I've recently been considering some challenges in expressing and communicating uncertainty. I've also been thinking about the critical elements of reproducible data science workflows that I need to emphasize for my students and colleagues. I find myself once again reminded of how difficult it can be to understand uncertai...
Read more

Conformal Prediction: Simple Model-Agnostic Prediction with Statistically Guaranteed Error Rates

This Blog Post about Conformal Prediction © 2023 by Lynd Bacon and Loma Buena Associates, is licensed under CC BY-SA 4.0    TL;DR? Here’s the BLUF:  "Conformal prediction methods can provide you with statistically valid certainty guarantees regarding the likely values of predictions using “new” data, data that you haven’t observed outcomes for, or that you just didn’t use for training and validating models." A simple classification example with Python code snippets follows below, along with some suggested follow-up resources.    Suppose you’ve trained, tuned, and validated a predictive model that you will use to predict outcomes for new “objects,” new cases or observations. Your model might be simple regression or classification model, an ensemble model of some sort, or a deep neural network. Your objects might be customers and their responses or their types, drug molecules and their efficacy, or any other sort of thing that you use a model to p...
Read more