Reproducability

• From this blog, I got aware of this reproducibility issue in RL i.e execution of same alogrithm in same enironment gives different results each time. It might be due different initial conditions, seeds etc for e.g. issues faced when reproducing a deep RL paper by Matthew Rahtz.

• For which the author proposes some statistical tests and he has a written a paper about this.

May be I have to have a look on it later.

• ACL has been already implemented by these guys.

• Some Eng resources I got,

  1. Nuts and Bolts of Deep RL Experimentation by John Schulman
  2. Notes by Falcon
  3. Nuts and Bolts of Deep RL Experimentation by John Schulman
  4. Notes by Falcon

micro-ROS

Today I did a hello world in micro-ROS. micro-ROS is used for interfacing ROS with resource constrained embedded devices. I had bought an ESP32-WROOM board since micro-ROS supports ESP, I thought of trying it and followed this post. In which I did,

• I had compiled the int32_publisher example using idf.py(provided by ESP) and flashed it to my ESP board.

• Then ran a micro-ROS agent(docker container) on my laptop and which recieved messages from ESP.

Basically we have to write a C code using ESP,micro-ROS and RTOS(FreeRTOS) libraries which then can be compiled & flashed into ESP and then it works accordingly. I had this issue with specifying the port for the agent.

Have to go through the rclc API.

I am gonna work on some project like with FreeRTOS & micro-ROS ?

Bayesian Optimization

Bayesian optimization is a powerful strategy for finding the extrema of objective functions that are expensive to evaluate. It is particularly useful when these evaluations are costly, when one does not have access to derivatives, or when the problem at hand is non-convex.

The Bayesian Optimization algorithm can be summarized as follows:

1. Select a Sample by Optimizing the Acquisition Function.
2. Evaluate the Sample With the Objective Function.
3. Update the Data and, in turn, the Surrogate Function.
4. Go To 1.

• It uses a
  • Surrogate function - that approximates the relationship between I/O data of the sample. There are many ways to model, one of the ways is to use Random Forest/Gaussian Process (GP, with many different kernels) i.e here we’re kind of approximating the objective function such that it can be easily sampled.
  • Acquisition function - It gives a sample that is to evaluated by the objective function. It is found by optimizing this function by various methods and it balances exploitation and exploration (E&E)[1].
• It is highly used in Tuning of Hyperparameters e.g. Optuna,HyperOpt.

Optuna

• API Reference

• This video gives a good intro.

I am trying to use it for HPO of lunar lander environment, initally results weren’t that good. I think it’s because of not giving a proper intreval i.e a large intreval that won’t result in a good choice of HP. May be I have to give try other ways to make it work.

Paper Reading

• I’m reading this paper, “Improving Environment Robustness of Deep Reinforcement Learning Approaches for Autonomous Racing Using Bayesian Optimization-based Curriculum Learning”, which I saw from ArXiV (in CS.RO) today, where they propose a method for automated curriculum selection which lead me to know about Bayesian Optimization.

• Curriculum Learning is about spoon feeding different environments for agent exploration, so that we can get a robust policy.

References

1. https://machinelearningmastery.com/what-is-bayesian-optimization/

Here’s a Python program to accomplish this task. The script will:

1. Go to a specified source folder.
2. List all files with the .mp3 extension.
3. Extract metadata date and compare it with the user-provided date.
4. Move files that match the criteria to a specified destination folder.

To achieve this, you’ll need to install the mutagen library for handling MP3 metadata. You can install it using pip install mutagen.

Here’s the Python script:

import os
import shutil
from datetime import datetime
from mutagen.mp3 import MP3

def get_mp3_date(file_path):
    try:
        audio = MP3(file_path)
        if audio.tags is not None:
            return datetime.fromtimestamp(audio.info.pprint())
    except Exception as e:
        print(f"Error reading {file_path}: {e}")
    return None

def move_files_by_date(source_folder, destination_folder, user_date):
    if not os.path.exists(destination_folder):
        os.makedirs(destination_folder)
    
    for root, _, files in os.walk(source_folder):
        for file in files:
            if file.lower().endswith('.mp3'):
                file_path = os.path.join(root, file)
                file_date = get_mp3_date(file_path)
                
                if file_date and file_date.date() == user_date:
                    shutil.move(file_path, os.path.join(destination_folder, file))
                    print(f"Moved: {file}")

if __name__ == "__main__":
    source_folder = "path/to/source/folder"
    destination_folder = "path/to/destination/folder"
    user_date = datetime.strptime("2023-08-06", "%Y-%m-%d").date()

    move_files_by_date(source_folder, destination_folder, user_date)

Instructions to Run the Script:

1. Install Dependencies: Ensure you have Python installed. Install the mutagen library using pip install mutagen.
2. Update Paths and Date:
   • Replace "path/to/source/folder" with the path to your source folder containing the MP3 files.
   • Replace "path/to/destination/folder" with the path to your destination folder where you want to move the files.
   • Replace "2023-08-06" with the user-specified date you want to compare against.
3. Run the Script: Save the script as a .py file and run it using a Python interpreter.

The script will scan the source folder, check the metadata date of each MP3 file, and move files that match the user-specified date to the destination folder.

Objective:

Develop a Python program that:

1. Scans a specified source folder for MP3 files.
2. Extracts the metadata date from each MP3 file.
3. Compares this date to a user-provided date.
4. Moves files with matching dates to a destination folder.

Required Libraries:

• os: To navigate the file system.
• shutil: To move files between directories.
• datetime: To handle date operations.
• mutagen: To read metadata from MP3 files.

Step-by-Step Process:

1. Import Necessary Libraries:
   • os for navigating directories.
   • shutil for moving files.
   • datetime for handling date comparisons.
   • mutagen.mp3 for reading MP3 metadata.
2. Define Function to Extract MP3 Metadata Date:
   • Use mutagen.mp3.MP3 to read the MP3 file.
   • Extract the date from the metadata if available.
   • Return the date as a datetime object.
3. Define Function to Move Files:
   • Navigate the source directory to find MP3 files.
   • For each MP3 file, extract the metadata date.
   • Compare the metadata date with the user-provided date.
   • If dates match, move the file to the destination folder.
4. Main Execution Block:
   • Define the source and destination folders.
   • Define the user-provided date for comparison.
   • Call the function to move files based on the date comparison.

To install the mutagen library using pip, follow these steps:

Steps to Install mutagen Using pip:

1. Open a Command Prompt or Terminal:
   • On Windows: Press Win + R, type cmd, and press Enter.
   • On macOS/Linux: Open the Terminal from the Applications menu or use the shortcut Ctrl + Alt + T (Linux) or Cmd + Space and type “Terminal” (macOS).
2. Ensure pip is Installed:
   • Check if pip is installed by running:

pip --version

If pip is not installed, you can install it by following the official pip installation guide.

3.Install mutagen:

• Run the following command to install the mutagen library:

pip install mutagen

Example:

On Windows/MacOS/Linux:

open in command prompt/Terminal:

pip install mutagen

Verifying the Installation:

After the installation is complete, you can verify that mutagen is installed by running a simple Python command:

import mutagen
print(mutagen.version)

You can run this command in a Python interpreter or save it in a .py file and execute it. If there are no errors and the version prints correctly, mutagen is installed successfully.

I try the above steps but have facing some error I have discussed about this particular program