Category: ML/AI/DL

Feature Engineering

June 10th, 2023
It involves converting raw data from one or more sources into features using domain knowledge and statistics that can be used for training ML models.

A feature engineering lifecycle has three steps –
1. Feature Selection
2. Feature Creation
3. Feature transformation
Feature Selection

Identify required features from training dataset and filter out the redundant and unnecessary ones from the training dataset. It helps reduce feature dimensionality, which produces small dataset and so speeds up ML model training.

One of the techniques used for feature selection is feature importance score. It tells how much a feature contributes to final model with respect to other features. So we can filter out features with feature importance score of 0 and analyze remaining ones based on score whether to keep or filter.

Feature Creation

Combine existing features into new features or combine attributes into new attributes. These new features helps models produce more accurate predictions

Feature transformation

Feature transformation is calculating missing features using techniques like imputation. It also includes scaling numerical values using techniques like standardization and normalization. And converting non numerical features into numerical features so that model can make sense of these non numerical features using techniques like one hot encoding.

Feature Engineering pipeline

Once the feature selection/creation is done, data set is split into training dataset and test data set. Training dataset is further split into training and validation dataset. Validation dataset is used to evaluate model and tune hyperparameters.

Thanks for stopping by! Hope this gives you a brief overview in to feature engineering. Eager to hear your thoughts and chat, please leave comments below and we can discuss.
Statistical Bias – What is it? | What causes it in a ML dataset?

June 4th, 2023
What is Statistical Bias?

A dataset is said to be biased if it cannot completely and accurately represent the underlying problem space.

Tendency of statistics is to either underestimate or overestimate a metric causes bias. When it comes to training data, there could be more values for some fields while very few for other fields. Think in database table, certain columns are sparse while others are dense.

Biased datasets could cause biased models, which in turn could result in biased analysis and affecting business decisions which would also be biased.

Example – Product review dataset. More reviews for product A than for product B and product C. When we would run sentiment analysis on this dataset, our model would be accurately able to predict sentiments towards product A then towards product B or product C. We can see our model is heavily biased towards product A.

Causes of Statistical Bias
1. Activity Bias – Biases exists in human generated content specially social media content. Very small percent of population is on these platforms. So data collected over years on these platforms doesn’t represent the whole population. Let’s consider facebook, there is still lot of people who don’t use it/use it very rarely, any data inferred from the data collected from facebook would be biased towards the opinions/like/dislikes of only that sector of the population.
2. Societal Bias – Biases existing in human generated content in addition to social media content because of preconceived notions and conscious biases. An example can be the bias in existing voice assistants. The voice assistants have been trained only generally with US and british accents of english. Say, person A is in US but for him english is second language and has an accent. He is using voice assistant with his device set to locale, that voice assistant will have difficulty in understanding his words.
3. Selection Bias – Feedback loop. Bias introduced by the ML system itself. For example, a ML system gives user few options to choose from, and then uses user’s selection to train the model. This introduces feedback loops. Recently I ordered a sippy cup for my baby from Amazon, now whenever I open the app, in the section for items that can interest you (system recommended), I see different sippy cups/bottles. Here based on the items I am buying, system has started recommending similar items on me, although that could have been one off purchase for that item from Amazon.
4. Data drift – Happens after model is trained and deployed. Happens when data distribution that model is getting is different from the data distribution model was trained on.
  1. covariant drift – distribution of independent variables or features used by model significantly changes.
  2. prior probability drift – distribution of labels or targeted variables change.
  3. concept drift – relationship between features and labels change.
Thanks for stopping by! Hope this gives you a brief overview in to statistical bias. Eager to hear your thoughts and chat, please leave comments below and we can discuss.

Category: ML/AI/DL

Feature Engineering

Feature Selection

Feature Creation

Feature transformation

Feature Engineering pipeline

Statistical Bias – What is it? | What causes it in a ML dataset?

What is Statistical Bias?

Causes of Statistical Bias