Skip to content

Instantly share code, notes, and snippets.

@samyarmodabber
Last active February 28, 2023 21:44
Show Gist options
  • Save samyarmodabber/2d545870d69278cd9282704954c61a92 to your computer and use it in GitHub Desktop.
Save samyarmodabber/2d545870d69278cd9282704954c61a92 to your computer and use it in GitHub Desktop.

Chapter 1: Introduction

Interpretation of probability

The frequentist interpretation and Bayesian interpretation of probability are two philosophical and mathematical frameworks for understanding probability. They differ in their assumptions about the nature of probability, the role of data and evidence, and the interpretation of results.

Frequentist

The frequentist interpretation of probability defines probability as the long-run relative frequency of an event occurring in a large number of independent repetitions of a random experiment.

It does not allow for subjective beliefs or uncertainty in a proposition, but instead defines probability in terms of the observed frequency of an event. The frequentist interpretation is often used in statistical inference, where probabilities are associated with the likelihood of obtaining a certain data sample given a particular hypothesis or model.

Bayesian

The Bayesian interpretation of probability defines probability as a measure of subjective belief or uncertainty in a proposition, given available evidence or data.

It allows for subjective beliefs and uncertainty, and is often used in decision making, where probabilities are used to make optimal decisions based on available evidence.

It is based on the idea that probabilities can be updated in light of new evidence or data through Bayes' theorem, which relates the probability of a proposition before and after the data is observed.

Frequentist vs Bayesian

The key difference between the frequentist and Bayesian interpretations is that the frequentist interpretation treats probability as an objective property of the physical world, while the Bayesian interpretation treats probability as a subjective measure of belief.

The frequentist approach relies on the relative frequency of events observed in large samples, while the Bayesian approach incorporates prior beliefs and updates them based on observed data.

Overall, both interpretations have their strengths and weaknesses and are used in different contexts depending on the nature of the problem and the available data and evidence. The choice of interpretation often depends on the assumptions and values of the decision maker and the goals of the analysis.

Model uncertainty and Data uncertainty

Model uncertainty and data uncertainty are two types of uncertainty that can impact the accuracy and reliability of modeling and decision making.

They can have significant impacts on the accuracy and reliability of modeling and decision making, and they should be carefully considered and addressed.

They are critical for making reliable and accurate predictions and decisions based on modeling and data analysis.

Model uncertainty

Model uncertainty (epistemic uncertainty) refers to uncertainty that arises from the choice of model used to represent a system or process. It reflects the fact that different models can lead to different predictions or outcomes, and it is often difficult to determine which model is the "true" representation of the system.

Model uncertainty can arise from simplifying assumptions, errors in model specification or parameter estimation, and variability in model performance across different data sets or contexts.

Strategies for addressing model uncertainty include model averaging or model selection based on various criteria such as goodness of fit, predictive accuracy, or parsimony.

Data uncertainty

Data uncertainty (aleatoric uncertainty) refers to uncertainty that arises from the measurement or collection of data used to inform the model. It reflects the fact that data can be noisy, incomplete, or biased, and may not fully capture the complexity of the system being modeled.

It refers to uncertainty that is inherent in the system being modeled or observed, and cannot be reduced or eliminated even with perfect knowledge and understanding of the system.

Data uncertainty can arise from errors in measurement, missing data, sampling bias, and other sources of variability in the data.

Strategies for addressing data uncertainty include sensitivity analysis, robust modeling, and careful consideration of data quality and limitations.

Events

Understanding events is a fundamental concept in probability theory, and is used to calculate probabilities, make predictions, and make decisions in a wide range of fields, including finance, engineering, and medicine.

Definition (experiment, outcomes, event)

  • An experiment is any process that can be repeated multiple times under similar conditions.
  • The outcomes of an experiment can be classified into two types: possible and impossible.
  • A sample space is the set of all possible outcomes of an experiment.
  • An event is a subset of the possible outcomes of an experiment.

Mostly Sample space shows by $\Omega$ and events by capital letters. By definition, $A$ is an event if $A\in P\left( \Omega \right)$.

For example, in flipping a coin, the sample space is $\Omega=\lbrace heads,tails \rbrace = \lbrace H,T \rbrace$, because these are the only possible outcomes.

In rolling a six-sided die, the sample space is $\Omega =\lbrace 1, 2, 3, 4, 5, 6 \rbrace$, because these are the only possible outcomes.

Events can be classified based on their relationship to each other.

Definition (mutually exclusive, independent)

Two events are said to be

  • mutually exclusive if they cannot both occur at the same time.
  • independent if the occurrence of one event does not affect the probability of the other event occurring.

For example, in rolling a six-sided die, the events of rolling a 3 and rolling a 5 are mutually exclusive. and in flipping a coin twice, the events of getting "heads" on the first flip and getting "heads" on the second flip are independent.

Events can be combined using set operations, such as union, intersection, and complement. The union of two events is the set of outcomes that belong to either one or both of the events. The intersection of two events is the set of outcomes that belong to both events. The complement of an event is the set of outcomes that do not belong to the event.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment