Skip to content

Instantly share code, notes, and snippets.

@fenar
Created December 18, 2024 15:16
Show Gist options
  • Select an option

  • Save fenar/7bd46e86820b03528c3ba3a9324f126a to your computer and use it in GitHub Desktop.

Select an option

Save fenar/7bd46e86820b03528c3ba3a9324f126a to your computer and use it in GitHub Desktop.
RLHF

The Ultimate Reinforcement Learning Bible: Concepts, Code, and Applications

1. Introduction to Reinforcement Learning

1.1 What is Reinforcement Learning?

Imagine teaching a dog new tricks. You don't explicitly tell the dog exactly how to sit or roll over - instead, you reward good behavior with treats and perhaps gently discourage unwanted behavior. Over time, the dog learns what actions lead to treats and begins to make better decisions.

This is exactly how reinforcement learning works in the world of artificial intelligence!

Key Components:

  • Agent: The learner or decision-maker
  • Environment: The world in which the agent operates
  • State: The current situation
  • Action: What the agent can do
  • Reward: Feedback that tells the agent how good or bad an action was

Concrete Example: Maze Navigation Robot

class MazeNavigationEnvironment:
    def __init__(self, maze_layout):
        self.maze = maze_layout
        self.current_position = self.start_position
        self.goal_position = self.find_goal_position()
    
    def step(self, action):
        """
        Actions: 0 = Up, 1 = Right, 2 = Down, 3 = Left
        Returns: (next_state, reward, is_done)
        """
        # Calculate next position based on action
        if action == 0:  # Up
            next_pos = (self.current_position[0] - 1, self.current_position[1])
        elif action == 1:  # Right
            next_pos = (self.current_position[0], self.current_position[1] + 1)
        elif action == 2:  # Down
            next_pos = (self.current_position[0] + 1, self.current_position[1])
        else:  # Left
            next_pos = (self.current_position[0], self.current_position[1] - 1)
        
        # Reward mechanism
        if next_pos == self.goal_position:
            return next_pos, +10, True  # Reached goal
        elif self.is_wall(next_pos):
            return next_pos, -1, False  # Hit a wall
        else:
            # Encourage exploration and efficiency
            distance_to_goal = self.calculate_distance(next_pos, self.goal_position)
            return next_pos, -0.1 * distance_to_goal, False

1.2 The Learning Loop

  1. Observe the current situation (State)
  2. Choose an action
  3. Perform the action
  4. Receive feedback (Reward)
  5. Learn and improve

2. Mathematical Foundation: Markov Decision Process (MDP)

2.1 Core Mathematical Framework

The MDP consists of:

  • State Space (S): All possible situations
  • Action Space (A): All possible actions
  • Transition Function: How actions change states
  • Reward Function: How actions are evaluated
  • Discount Factor (γ): Importance of future rewards

Technical Implementation of MDP

class MDPFramework:
    def __init__(self, state_space, action_space):
        # Probability of transitioning between states
        self.transition_probability = {}
        
        # Reward for each state-action pair
        self.reward_function = {}
        
        # Discount factor (how much future rewards matter)
        self.gamma = 0.95
    
    def calculate_value_iteration(self, state):
        """
        Compute the value of a state using the Bellman equation
        V(s) = max[ R(s,a) + γ * Σ P(s'|s,a) * V(s') ]
        """
        max_value = float('-inf')
        for action in self.action_space:
            expected_future_value = 0
            for next_state in self.state_space:
                # Probability of reaching next state * value of next state
                transition_prob = self.transition_probability.get((state, action, next_state), 0)
                value_next_state = self.value_function.get(next_state, 0)
                
                expected_future_value += transition_prob * value_next_state
            
            # Calculate total value including immediate reward
            current_value = (self.reward_function.get((state, action), 0) + 
                             self.gamma * expected_future_value)
            
            max_value = max(max_value, current_value)
        
        return max_value

2.2 Value Function Equation

The core idea is to estimate the expected total reward from a given state:

V(s) = max_a [ R(s,a) + γ * Σ P(s'|s,a) * V(s') ]

3. Fundamental Learning Algorithms

3.1 Q-Learning: Learning Action Values

Q-Learning helps an agent learn the value of taking specific actions in different states.

class QLearningAgent:
    def __init__(self, state_space, action_space, learning_rate=0.1, discount_factor=0.99, exploration_rate=0.1):
        # Q-table to store state-action values
        self.Q = defaultdict(lambda: defaultdict(float))
        
        # Hyperparameters
        self.alpha = learning_rate  # Learning rate
        self.gamma = discount_factor  # Discount factor
        self.epsilon = exploration_rate  # Exploration rate
    
    def choose_action(self, state):
        # Epsilon-greedy action selection
        if random.random() < self.epsilon:
            return random.choice(list(self.Q[state].keys()))
        else:
            return max(self.Q[state], key=self.Q[state].get)
    
    def learn(self, state, action, reward, next_state):
        # Q-Learning update rule
        best_next_action = max(self.Q[next_state], key=self.Q[next_state].get)
        
        # Q(s,a) = Q(s,a) + α[r + γ * max(Q(s',a')) - Q(s,a)]
        current_q = self.Q[state][action]
        max_next_q = self.Q[next_state][best_next_action]
        
        new_q = current_q + self.alpha * (
            reward + self.gamma * max_next_q - current_q
        )
        
        self.Q[state][action] = new_q

3.2 Policy Gradient: Learning Policies Directly

Policy Gradient methods learn a policy that directly maps states to actions.

class PolicyGradientAgent(nn.Module):
    def __init__(self, state_dim, action_dim):
        super().__init__()
        # Neural network policy
        self.policy_network = nn.Sequential(
            nn.Linear(state_dim, 64),
            nn.ReLU(),
            nn.Linear(64, 64),
            nn.ReLU(),
            nn.Linear(64, action_dim),
            nn.Softmax(dim=-1)
        )
    
    def forward(self, state):
        return self.policy_network(state)
    
    def compute_loss(self, states, actions, returns):
        # Compute policy gradients
        log_probs = torch.log(self.forward(states).gather(1, actions))
        loss = -(log_probs * returns).mean()
        return loss

4. Advanced Reinforcement Learning Techniques

4.1 Monte Carlo Tree Search (MCTS)

A sophisticated search algorithm that builds a tree of possible future states and simulates potential outcomes.

class MCTSNode:
    def __init__(self, state, parent=None):
        self.state = state
        self.parent = parent
        self.children = {}
        self.visits = 0
        self.total_reward = 0
    
    def uct_score(self, exploration_constant=1.414):
        # Upper Confidence Bound for Trees (UCT)
        if self.visits == 0:
            return float('inf')
        
        exploitation_score = self.total_reward / self.visits
        exploration_bonus = exploration_constant * math.sqrt(
            math.log(self.parent.visits) / self.visits
        )
        
        return exploitation_score + exploration_bonus
    
    def select_best_child(self):
        return max(self.children.values(), key=lambda child: child.uct_score())

5. Reinforcement Learning from Human Feedback (RLHF)

5.1 Direct Preference Optimization (DPO)

A method to directly learn from human preferences without explicit reward modeling.

class DPOTrainer:
    def __init__(self, model, reference_model):
        self.model = model
        self.reference_model = reference_model
    
    def compute_dpo_loss(self, preferred_output, rejected_output, beta=0.1):
        # Compute log probabilities
        preferred_log_prob = torch.log_softmax(preferred_output, dim=-1)
        rejected_log_prob = torch.log_softmax(rejected_output, dim=-1)
        
        # Compute preference probability
        logits_diff = preferred_log_prob - rejected_log_prob
        
        # DPO objective
        preference_loss = -F.logsigmoid(beta * logits_diff).mean()
        
        # Regularization to prevent model from deviating too much
        kl_divergence = self.compute_kl_divergence(
            self.model, 
            self.reference_model
        )
        
        return preference_loss + kl_divergence

6. Practical Considerations

6.1 Exploration Strategies

Balancing exploration (trying new things) and exploitation (using known good actions).

class ExplorationStrategy:
    def __init__(self, action_space, initial_epsilon=1.0, final_epsilon=0.1, decay_steps=10000):
        self.action_space = action_space
        self.epsilon = initial_epsilon
        self.final_epsilon = final_epsilon
        self.decay_steps = decay_steps
    
    def get_action(self, q_values, current_step):
        # Epsilon-decay strategy
        self.epsilon = max(
            self.final_epsilon, 
            self.initial_epsilon - (current_step / self.decay_steps)
        )
        
        # Exploration vs exploitation
        if random.random() < self.epsilon:
            return random.choice(self.action_space)
        else:
            return np.argmax(q_values)

7. Future Directions and Research Frontiers

Current cutting-edge research areas:

  1. Multi-agent Reinforcement Learning
  2. Meta-learning
  3. Safe Reinforcement Learning
  4. Hierarchical RL
  5. Offline Reinforcement Learning

Conclusion: Key Insights

  1. Complexity Management: Balance sophisticated models with interpretability
  2. Efficient Learning: Develop techniques to learn from limited data
  3. Generalization: Create adaptive agents
  4. Safety: Implement constraints to prevent harmful exploration

Recommended Learning Path

  1. Master fundamental RL algorithms
  2. Implement algorithms from scratch
  3. Experiment with complex environments
  4. Study recent research papers
  5. Contribute to open-source RL libraries

Pro Tips for RL Practitioners:

  • Start with simple environments
  • Visualize your agent's learning process
  • Understand the mathematical foundations
  • Experiment constantly
  • Don't be afraid to fail and learn from mistakes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment