Show notes: ocdevel.com/mlg/1. MLG teaches the fundamentals of machine learning and artificial intelligence. It covers intuition, models, math, languages, frameworks, etc. Where your other ML resources provide the trees, I provide the forest. Consider MLG your syllabus, with highly-curated resources for each episode's details at ocdevel.com. Audio is a great supplement during exercise, commute, chores, etc.

• MLG, Resources Guide
• Gnothi (podcast project): website, Github

• "Middle" level overview (deeper than a bird's eye view of machine learning; higher than math equations)
• No math/programming experience required

Who is it for

• Anyone curious about machine learning fundamentals
• Aspiring machine learning developers

Why audio?

• Supplementary content for commute/exercise/chores will help solidify your book/course-work

What it's not

• News and Interviews: TWiML and AI, O'Reilly Data Show, Talking machines
• Misc Topics: Linear Digressions, Data Skeptic, Learning machines 101
• iTunesU issues

Planned episodes

• What is AI/ML: definition, comparison, history
• Inspiration: automation, singularity, consciousness
• ML Intuition: learning basics (infer/error/train); supervised/unsupervised/reinforcement; applications
• Math overview: linear algebra, statistics, calculus
• Linear models: supervised (regression, classification); unsupervised
• Parts: regularization, performance evaluation, dimensionality reduction, etc
• Deep models: neural networks, recurrent neural networks (RNNs), convolutional neural networks (convnets/CNNs)
• Languages and Frameworks: Python vs R vs Java vs C/C++ vs MATLAB, etc; TensorFlow vs Torch vs Theano vs Spark, etc

Try a walking desk to stay healthy while you study or work!

Show notes at ocdevel.com/mlg/3.

This episode covers four major philosophical topics related to artificial intelligence. The purpose is to give broader context to why AI matters, before moving into technical details in later episodes.

AI is automating not just simple tasks like data entry or tax prep, but also high-skill jobs such as medical diagnostics, surgery, and creative work like design, music, and art. There are two common reactions:

• Fear: Concern over job displacement, similar to past economic shifts like the agricultural and industrial revolutions.
• Is your job safe?
• Optimism: Automation may lead to more comfortable living conditions and economic structures like Universal Basic Income. New job types could emerge, as they have in past transitions.

The singularity refers to a point of runaway technological growth, where AI becomes capable of improving itself recursively. This concept is tied to "artificial general intelligence" and "seed AI"—systems that not only perform tasks but create better versions of themselves. The idea is that this could trigger extremely rapid change, possibly representing a new phase of evolution beyond humanity.

I explore whether consciousness can emerge from machines. Since the brain is a physical machine and consciousness arises from it, it's possible that artificial systems could develop similar properties. Related ideas:

• Qualia: Subjective experiences.
• Functionalism: If something behaves like it’s conscious, it may be conscious.
• Turing Test: If a machine is indistinguishable from a human in conversation, it passes the test.

I discuss scenarios where AI causes harm not through malevolence but through poorly defined objectives. One example is the "paperclip maximizer" thought experiment, where an AI tasked with maximizing paperclip production might consume all resources to do so. This has led some public figures to raise concerns about AI safety. I don't share the same level of concern, but the topic is worth being aware of.

• Ray Kurzweil, The Singularity is Near
• Ray Kurzweil, How to Create a Mind
• Daniel Dennett, Consciousness Explained
• Nick Bostrom, Superintelligence
• The Great Courses, Philosophy of Mind, Brain, Consciousness, and Thinking Machines

In the next episode, I begin covering the technical foundations of machine learning, starting with supervised, unsupervised, and reinforcement learning.

Machine learning consists of three steps: prediction, error evaluation, and learning, implemented by training algorithms on large datasets to build models that can make decisions or classifications. The primary categories of machine learning algorithms are supervised, unsupervised, and reinforcement learning, each with distinct methodologies for learning from data or experience.

• Notes and resources at ocdevel.com/mlg/4
• Try a walking desk stay healthy & sharp while you learn & code

• Artificial intelligence includes subfields such as reasoning, knowledge representation, search, planning, and learning.
• Learning connects to other AI subfields by enabling systems to improve from mistakes and past actions.

• The machine learning process follows three steps: prediction (or inference), error evaluation (or loss calculation), and training (or learning).
• In an example such as predicting chess moves, a move is made (prediction), the error or effectiveness of that move is measured (error function), and the underlying model is updated based on that error (learning).
• This process generalizes to real-world applications like predicting house prices, where a model is trained on a large dataset with many features.

• Datasets used for machine learning are typically structured as spreadsheets with rows as examples (e.g., individual houses) and columns as features (e.g., number of bedrooms, bathrooms, square footage).
• Features are variables used by algorithms to make predictions and can be numerical (such as square footage) or categorical (such as "is downtown" yes/no).
• The algorithm processes input data, learns the appropriate coefficients or weights for each feature through algebraic equations, and forms a model.
• The combination of the algorithm (such as code in Python or TensorFlow) and the learned weights forms the model, which is then used to make future predictions.

• After the initial training on a dataset, models can be updated incrementally with new data (called online learning).
• When new outcomes are observed that differ from predictions, this new information is used to further train and improve the model.

• Machine learning algorithms are broadly grouped into three categories: supervised, unsupervised, and reinforcement learning.
  • Supervised learning uses labeled data, where the model is trained with known inputs and outputs, such as predicting prices (continuous values) or classes (like cat/dog/tree).
  • Unsupervised learning finds similarities within data without labeled outcomes, often used for clustering or segmentation tasks such as organizing users for advertising.
  • Reinforcement learning involves an agent taking actions in an environment to achieve a goal, receiving rewards or penalties, and learning the best strategies (policies) over time.

• Regression algorithms like linear regression are commonly used supervised learning techniques to predict numeric outcomes.
• The process is rooted in algebra and particularly linear algebra, where matrices represent datasets and the algorithm solves for optimal coefficient values.
• The model’s equation generated during training is used for making future predictions, and errors from predictions guide further learning.

• MachineLearningMastery.com: Accessible articles on ML basics.
• Podcast’s own curated learning paths: ocdevel.com/mlg/resources.
• The book "The Master Algorithm" offers an introductory and audio format overview of foundational machine learning algorithms and concepts.

Try a walking desk to stay healthy while you study or work!

Full notes at ocdevel.com/mlg/6

• Individuals may engage with machine learning for self-education, as a hobby, or to enter the industry professionally.
• Use a combination of resources, including podcasts, online courses, and textbooks, for a comprehensive self-learning plan.

• MOOCs, or Massive Open Online Courses, offer accessible education.
• Key platforms: Coursera and Udacity. Coursera is noted for standalone courses; Udacity offers structured nanodegrees.
• Udacity nanodegrees include video content, mentoring, projects, and peer interaction, priced at $200/month.

• Udacity nanodegrees are currently not widely recognized or respected by employers.
• Emphasize building a robust portfolio of independent projects to augment qualifications in the field.

• Master’s Degrees:
• Valued by employers, provide an edge in job applications.
• Example: Georgia Tech's OMSCS (Online Master’s of Science in Computer Science) offers a cost-effective ($7,000) online master’s program.
• PhD Programs:
• Embark on a PhD for in-depth research in AI rather than industry entry. Program usually pays around $30,000/year.
• Compare industry roles (higher pay, practical applications) vs. academic research (lower pay, exploration of fundamental questions).

• Prioritize building a substantial portfolio of projects to bypass formal degree requirements and break into industry positions.
• Consider enriching your qualifications with a master's degree, or eventually pursue a PhD if deeply interested in pioneering AI research.

• See online discussions about degrees/certifications: 1 2 3 4

The logistic regression algorithm is used for classification tasks in supervised machine learning, distinguishing items by class (such as "expensive" or "not expensive") rather than predicting continuous numerical values. Logistic regression applies a sigmoid or logistic function to a linear regression model to generate probabilities, which are then used to assign class labels through a process involving hypothesis prediction, error evaluation with a log likelihood function, and parameter optimization using gradient descent.

• Notes and resources at ocdevel.com/mlg/7
• Try a walking desk - stay healthy & sharp while you learn & code

• Supervised learning consists of two main tasks: regression and classification.
• Regression algorithms predict continuous values, while classification algorithms assign classes or categories to data points.

• Logistic regression is a classification algorithm, despite its historically confusing name.
• The algorithm determines the probability that an input belongs to a specific class, using outputs between zero and one.

• The process starts by passing inputs through a linear regression function, then applying a logistic (sigmoid) function to produce a probability.
• For binary classification, results above 0.5 usually indicate a positive class (for example, “expensive”), and results below 0.5 indicate a negative class (“not expensive”).
• Multiclass problems assign probabilities to each class, selecting the class with the highest probability using the arg max function.

• An example uses a spreadsheet of houses with features like square footage and number of bedrooms, labeling each as "expensive" (1) or "not expensive" (0).
• Logistic regression uses the spreadsheet data to learn the pattern that separates expensive houses from less expensive ones.

• The algorithm follows three steps: predict (infer a class), evaluate error (calculate how inaccurate the guesses were), and train (refine the underlying parameters).
• Predictions are compared to actual data, and the difference (error) is calculated via a log likelihood function, which accounts for how confident the prediction was compared to the true value.
• Model parameters (theta values) are updated using gradient descent, which iteratively reduces the error by adjusting these values based on the derivative of the error function.

• The hypothesis function is the sigmoid or logistic function, with the formula: 1 / (1 + e^(-theta^T x)), where theta represents the parameters and x the input features.
• The error function (cost function) for logistic regression uses log likelihood, aggregating errors over all data points to guide model learning.

• Logistic regression finds a "decision boundary" on the graph (S-curve) that best separates classes such as "expensive" versus "not expensive."
• When the architecture requires a proper probability distribution (sum of probabilities equals one), a softmax function is applied to the outputs, but softmax is not covered in this episode.

• Machine learning architectures are highly compositional, with functions nested within other functions - logistic regression itself is a function of linear regression.
• This composability underpins more complex systems like neural networks, where each “neuron” can be seen as a logistic regression unit powered by linear regression.

• Understanding logistic and linear regression forms the foundation for approaching advanced areas of machine learning such as deep learning and neural networks.
• The concepts of prediction, error measurement, and iterative training recur in more sophisticated models.

• The episode recommends the Andrew Ng Coursera course for deeper study into these concepts and details, especially for further exploration of multivariate regression and error functions.

Try a walking desk to stay healthy while you study or work!

Full notes at ocdevel.com/mlg/8

• Linear Algebra: Essential for matrix operations; analogous to chopping vegetables in cooking. Every step of ML processes utilizes linear algebra.
• Statistics: The hardest part, akin to the cookbook; supplies algorithms for prediction and error functions.
• Calculus: Used in the learning phase (gradient descent), similar to baking; it determines the necessary adjustments via optimization.

• Recommendation: Learn the basics of machine learning first, then dive into necessary mathematical concepts to prevent burnout and improve appreciation.

• MOOCs: Khan Academy - Offers Calculus, Statistics, and Linear Algebra courses.
• Textbooks: Commonly recommended books for learning calculus, statistics, and linear algebra.
• Primers: Short PDFs covering essential concepts.

• The Great Courses: Offers comprehensive video series on calculus and statistics. Best used as audio for supplementing primary learning. Look out for "Mathematical Decision Making."

• Tensor: General term for any dimension list; TensorFlow from Google utilizes tensors for operations.
• Efficient computation using SimD (Single Instruction, Multiple Data) for vectorized operations.

• Gradient descent used for minimizing loss function, known as convex optimization. Recognize keywords like optimization in calculus context.