47th issue! If you missed them, you can read the previous issues of my monthly A.I. & Machine Learning newsletter here.
Hey there, Daniel here.
I’m an A.I. & Machine Learning Engineer who also teaches the following beginner-friendly machine learning courses:
I also write regularly about A.I. and machine learning on my own blog as well as make videos on the topic on YouTube.
Since there's a lot going on, the utmost care has been taken to keep things to the point.
Enough about me! You're here for this month's A.I. & Machine Learning Monthly Newsletter.
Typically a 500ish (+/-1,000ish, usually +) word post detailing some of the most interesting things on machine learning I've found in the last month.
My brother and I have been working on a food education app called Nutrify for a while. And now version 1.0 is live!
Nutrify allows you to take a photo of food and learn about it.
If you’ve done either of the ZTM TensorFlow or PyTorch courses, you would’ve worked on the Food Vision projects.
Nutrify is an extension of Food Vision and taking it to a fully deployed and functional application.
Nutrify is quick because it runs all the computer vision on device (the models are built with PyTorch and converted to Apple’s CoreML format).
For now, Nutrify can identify 420 foods (& mostly foods available in Australia 🇦🇺) but we’ll be adding many more in the future.
I’ve personally unlocked 333/420 foods with the app so far.
Let me know if you manage to beat it!
Nutrify: The Food App live on the App Store — take a photo of food and learn about it! If you’ve got an iPhone with iOS 16.0+, search “nutrify” on the App Store and look for the pineapple.
Keras 3.0 is one of the biggest releases in machine learning in the past decade.
But now the answer truly is “both if you want to”.
Because Keras 3.0 lets you use JAX, TensorFlow OR PyTorch as a backend.
So you can build a model in Keras and use it across any of the major frameworks.
A huge milestone for the machine learning community!
The example code snippet shows using PyTorch as the backend.
import numpy as np
os.environ["KERAS_BACKEND"] = "pytorch"
# Note that Keras should only be imported after the backend
# has been configured. The backend cannot be changed once the
# package is imported.
# Keras code will run with PyTorch as the backend
An embedding is a useful way to represent a data sample.
For example, if you have a sentence of text, you can use an embedding to represent it numerically. The same for images, audio files and other forms of data.
That numerical representation can then be used to compare it to other numerical representations.
This process is very achievable for a small number of samples (e.g. 100 to 10,000 and even 100,000).
However, once you start getting into the millions and billions of data samples, you’l likely want to create some kind of index on those samples to save on comparison time.
The workflow goes:
Data → Turn data into embeddings (choose algorithm) → Turn embeddings into index (choose index) → Search index to compare embeddings
An index is like a reference.
A flat index would mean you would search and compare one target sample to every other sample.
However, this could take quite a while if you have millions (or billions) of samples.
Instead, you can use an algorithm such as Hierarchical Navigable Small Worlds (HNSW) to create an index on your embeddings.
HNSW is a form of approximate search, meaning it won’t give you 100% of the best results (a flat index will), however, it will be much faster than searching over every sample.
Since I’ve been working with embeddings on a larger scale, I’ve been learning more and more about index creation algorithms like HNSW.
And I’ve found the following collection of resources invaluable:
A comparison of different indexes to use for embeddings and their results and runtimes. Source: Pinecone blog.
If you want to see an example of a workflow to use embeddings to build an image/text search system, Multimodal Retrieval with Text Embedding and CLIP Image Embedding for Backyard Birds by Wenqi Glantz is a fantastic example.
Given a question about a bird, the system will turn the text into an embedding, find a relevant resource in a database and then return an image along with an answer.
A very cool project idea that would be fun to replicate except with a different data source.
Example of using a multi-modal QA model and it returning text and image based on a text query. Source: Wenqi Glantz blog.
The Llama 2 Large Language Models (LLMs) from Meta are some of the best performing open-source LLMs available.
When they were first released, they were a bit of a challenge to get setup.
However, this has changed, thanks to following the resources:
Example of running llamafile locally on my MacBook Pro M1 Pro. I was shocked at how easy it was to setup and get going. It’s fast too.
A free book on deep learning formatted to be read on a phone screen for those with a STEM (science, technology, engineering and mathematics) background.
I’ve got this bookmarked for my holiday reading!
A few pages from The Little Book of Deep Learning by François Fleuret. Source: fleuret.org.
A free course consisting of 12 lessons on generative AI for absolute beginners by Microsoft. All materials are available on GitHub.
Overview of a modern day LLM-based application by GitHub. Like many machine learning apps, there’s a few more moving parts than just the model. Source: GitHub blog.
The AI Conference was held recently and two talks (there were many) I picked out were:
An overview of image classification, object localization (one object) and object detection (multiple objects). Source: Slide from Andrew Ng’s video on object localization.
What a massive month for the ML world in November!
As always, let me know if there's anything you think should be included in a future post.
In the meantime, keep learning, keep creating, keep dancing.
See you next month,
By the way, I'm also an instructor with Zero To Mastery Academy teaching people Machine Learning & AI in the most efficient way possible. You can see a few of our courses below or check out all Zero To Mastery courses.