80th issue! If you missed them, you can read the previous issues of my monthly A.I. & Machine Learning newsletter here.
Hey everyone!
Daniel here, I’m a machine learning engineer who teaches the following beginner-friendly machine learning courses:
- Complete Machine Learning and Data Science Bootcamp: Zero to Mastery
- TensorFlow for Deep Learning: Zero to Mastery
- PyTorch for Deep Learning: Zero to Mastery
- [NEW] 🤗 Machine Learning with Hugging Face Bootcamp: Zero to Mastery
I also write regularly about machine learning on my own blog as well as make videos on the topic on YouTube.
Since there's a lot going on, the utmost care has been taken to keep things to the point.
Here's what you might have missed in September 2026 as an A.I. & Machine Learning Engineer... let's get you caught up!
My Work
- Nutrify 3.0 — My brother Josh and I recently released Nutrify 3.0. Nutrify can now track foods in any modality, photo, text, voice or barcode. And if Nutrify makes a mistake, you can just text to correct it. There’s also over 100 new food icons to collect and share on your images, new ways to save favourites and set custom goals for different days. You can even track recipes with a screenshot or photo! If you’re looking for a whole-food focused tracking app, check out Nutrify (or send it to a friend).
From The Internet
- Bad code is kudzu by Vicki Boykis — Kudzu is a plant that was brought to the US to stop soil erosion and then grew so aggressively it now covers sidewalks, houses and trees. Vicki argues app features behave the same way, easy to plant, hard to remove, and with AI making new code cheap to generate, the overgrowth comes faster.
“In the era where it’s easy to add stuff, leave yourself a plan to remove the features you added, particularly since it’s much easier than before, too.”
- Hugging Face Transformers now runs llama.cpp quants — You can now load GGUF checkpoints (the quantized format used by llama.cpp) straight into Transformers with
from_pretrained, starting with Qwen3.5 models on Apple Silicon. For example, Unsloth’s Qwen3.5-4B goes from 8.42 GB in BF16 to 2.74 GB at Q4_K_M, the recommended starting point. “We’re adding support for running GGUF models efficiently in transformers, so you can use checkpoints sized for your laptop’s memory through the familiar transformers APIs.” - Google DeepMind introduce SynthID Bio — Watermarking for AI-generated proteins, much like SynthID watermarks AI-generated images. In wet-lab testing, the watermarked protein designs matched the hit rate and binding affinity of unwatermarked versions.
- Google Research: Enabling teachers to create learning interactives with generative UI — Teachers pick a topic and learning objectives, and generative UI builds a guided, game-like simulation with levels, hints and feedback. There’s also a public library of 30+ learning interactives for STEM subjects. I wonder what this means for the future of online interactive courses, perhaps materials could be built in real-time as a teacher explains a lesson or concept.
- Case studies from the claude.dev blog:
- How we made claude.ai 3x faster in two weeks — Anthropic ran a two-week performance sprint with Claude in every Slack thread and merged 3,000+ changes without a customer-facing incident or rollback. For example, time to a typeable page on a fresh load of claude.ai went from 3.1 seconds to 0.55 seconds (p75). “Once Claude can measure something, it can make it faster.”
- Automating eval design and hillclimbing with Claude — Principles for designing evals and improving against them without fooling yourself, plus new
/claude-api build-evaland/claude-api hillclimbcommands that put them to work. For example,build-evalinterviews you, builds the eval inside your codebase and pauses for your approval at specific points.- Claude’s new auto eval tool by Hamel Husain — Review from Hamel, one of my favourite go-to people in the world of AI evals. A very good point inside: he’d hold off using it for now, until its primary step is getting you to look at your data and build the eval from there. “Make sure the tool helps you understand your data before choosing evals and inspect its judgments carefully.”
- Splash is an open-source inference engine by Inco which makes your local AI models go faster — Instead of supporting every model, Splash is built around a small set of models (Qwen3.6-35B-A3B and Qwen3.8-27B today) and specializes everything for them on Apple silicon. On a 48 GB M5 Pro it decodes 1.7x and 2.0x as fast as the next-fastest engine tested, for example 210 tokens/s on Qwen3.6-35B-A3B with short prompts. It’s powered by DFlash 2 speculative decoding: every supported model ships with its own trained draft model, which proposes a whole block of tokens in one pass for the main model to verify (DFlash 2 is close to 3x the speed of regular decoding with the same output).
- Three blog posts from TypeSafe co-founder Diogo Almeida:
- (KV) Cache Rules Everything Around Me — Replaying real agent trajectories shows that output tokens are only 9-18% of an agent’s bill, the rest is the model re-reading its own history. For example, at Claude Code’s reported 89% cache hit rate, the same workload costs $1.97M instead of $1.07M with perfect caching. “The biggest cost to devs of an agent is re-reading its own context, which costs the provider close to nothing to serve.”
- The Bitterest Lesson — Rich Sutton’s bitter lesson says compute beats clever algorithms, and Diogo argues data and picking the right task matter even more. For example, at OpenAI, GPT-2-sized models trained on the right task (following instructions) beat GPT-3. “The bitterest lesson in ML is that doing the right task > data > compute > algorithms.”
- Lies, Damned Lies, and Benchmarks — On the pitfalls of benchmarks and how public evals get “benchmaxxed”. For example, Meta tested 27 private variants of Llama 4 on LMArena before picking the best one.
Vision
- DeepSeek V4 vision encoders in timm — The 412M parameter vision encoders from DeepSeek-V4-Flash and DeepSeek-V4.1-Flash, ready to use as image feature extractors.
- Qwen 3 vision encoders in timm — Vision encoders (88M to 416M parameters) from the Qwen3-VL, Qwen3.5 and Qwen3.8 model families.
- RT-DETRv4 — Apache 2.0 real-time object detection model trained with distillation from a DINOv3 (ViT-B/16) teacher. The smallest version gets 49.8 AP on COCO at 273 FPS on a T4 GPU. I’ve been using a fine-tuned RT-DETRv4 model for on-device food detection in a Nutrify prototype and it’s working really well.
- Qwen-Image-2.1 — A unified text-to-image generation and editing model with 7B parameters in its visual generation component, including native transparent (RGBA) image generation. Research only license, competitive with Nano Banana 2.
LLMs + VLMs
- Xiaomi MiMo-V2.6 Pro and Flash as well as a distilled version to Qwen3.5-9B — All MIT licensed. Xiaomi streamed the RL training run live on their website: in under six days, Flash and Pro each completed 30 RL steps over roughly 750k trajectories, at a cost of about $0.85M and $2.62M. See the release blog post for more.
- Ovis-VL-Embedding-9B — Multimodal embeddings for text, images, visual documents and video in one representation space, initialized from Qwen3.5-9B.
- Paper — The Ovis-Embedding technical report, with state-of-the-art results on MMEB-v3, MMEB-v2, MVEB, MAEB and RTEB.
- LensVLM from Apple — A 9B VLM which reads long documents as compressed images of text, with 5x, 10x and 15x compression options. For example, in the paper LensVLM keeps accuracy close to the full-text baseline at 4.3x effective compression and outperforms text and visual compression baselines up to 10.1x. It “scans compressed images of text, then selectively expands only the relevant pages to their uncompressed form via learned tools”.
- TeleOCR, Apache 2.0 OCR model — A ~1.2B parameter VLM for document parsing that handles both digital and camera-captured documents in one model.
- VisionPsy open-source 460M VLM — Apache 2.0 VLM from Tether AI Research which beats other ~0.5B VLMs on 16 of 17 benchmarks.
- Used InfoSFT to focus on tokens which maximise capability learning rather than memorization.
- Flash version is much faster on-device due to reducing the number of image tokens that get used so it’s got better time to first token (TTFT) and time to last token (TTLT).
- ZDTaichu5.0-9B for spatial VLM — Combines Qwen3.5-9B + C-RADIOv4-H vision encoder. Leads spatial reasoning and agent benchmarks among the 10B-scale VLMs it’s compared against.
Voice
- Edge0 release Audio8-ASR-Infinite — Apache 2.0 streaming speech recognition model for Chinese and English. For example, it decodes 12.5 times per second with a configurable 240-560 ms transcription delay. “With our adapted vLLM build it transcribes unlimited-length audio 24/7 without drifting.”
Tabular
- NVIDIA release Kumo-Tabular — NVIDIA’s pretrained foundation model for tabular classification and regression. Pass labeled rows in as context and get predictions on new rows, for example the quickstart uses 300 labeled rows of a breast cancer dataset to predict the rest. See the structured-data-models library for a collection of table-related models you can use in your own projects.
Releases
- TypeSafe introduce Jev, a System One/decision model — TypeSafe’s first System One model makes fast, structured decisions that software can use directly, instead of generating text. For example, input tokens cost $0.042 per million, output is free and responses take 70-500 ms end-to-end. “Think of Jev as a frontier-intelligence function call: unstructured state in, typed probabilistic decisions out.”
- SpaceXAI releases Grok 4.7 — A new, larger base model trained for coding and knowledge work, served at the same price and speed as Grok 4.6.
- OpenAI DevDay 2026 recap — 20+ announcements, including dots, GPT-6.1 Sol, an Ultrafast tier (up to 8x faster token generation in Codex) and the Decisions API. The Decisions API is similar to Jev’s API: you define questions with a fixed set of answers, give it text or image context and get back answers to classify content, route requests or choose an agent’s next action. It’s powered by GPT-6 Luna and isn’t public yet (limited preview).
- OpenAI releases GPT-6 Astra + GPT-6.1 Sol — GPT-6.1 Sol is an upgrade to GPT-6 Sol that gets close to GPT-6 Astra for much less. For example, at maximum effort on Terminal-Bench Science it costs $5.47 per task compared to $23.80 for Astra. It “nearly matches GPT-6 Astra’s intelligence on agentic coding, computer use, and professional work at one-fifth of Astra’s standard input and output token prices”. In my personal experience, GPT-6 Astra and GPT-6.1 Sol have had very good improvements in vision, GPT-6.1 Sol gets about 90-99% of GPT-6 Astra’s bounding box capability but at a much lower price.
- OpenAI introduce dots as an always on AI assistant — Always-on agents powered by GPT-6 Astra, each with their own cloud computer, which you can reach through ChatGPT, Slack or Teams.
- Claude Opus 5.5 and Claude Sonnet 5.5 — The first two models in the Claude 5.5 family. On the Artificial Analysis Intelligence Index, Sonnet 5.5 scores 56 compared to 38 for Sonnet 5 and outputs 139 tokens per second compared to 80 (both at max effort). Opus 5.5 “performs at the level of Claude Fable 5.1 on most work and costs 40% less to run than Opus 5”.
- Google releases Gemini 4 Argon — Google’s new frontier model for coding, enterprise knowledge work and cyber defense, with an introductory price of $2 per million input tokens and $10 per million output tokens. Not publicly available yet. Argon is rolling out to trusted cyber defenders through Google’s Fairwind Program first.
Open/Replica Jev’s
- Jev’s Architecture Unmasked — Blog post detailing speculations behind Jev’s architecture. “I probed Jev with 10,000 API calls to work out roughly how it’s built, and why most of the grifter takes on X are completely wrong.”
Production level:
- Coming soon: OpenAI Decisions API — Not publicly released yet, it’s in limited preview with a broad release planned in the coming days. Ask questions with a finite set of pre-defined answers about text or image context.
- Liquid AI decision models (d1) — Liquid AI’s first decision model, for classification, routing and scoring in a single call. “Instead of generating text token by token, a decision model evaluates a situation and returns calibrated probabilities across a fixed set of outcomes in a single call with zero generated tokens.”
Self-host/open-source production level:
- SGLang Decision Models — No special checkpoint needed, SGLang reads option probabilities from any chat model’s next-token scores at the answer position. “/v1/decisions turns a chat model into a decision model.”
- GLiNER 2.5 Decide — A 340M English classification model, pass any label set at call time and get results in a single forward pass.
- Laya, ~400M small decisions model — Multilingual (100+ languages) decision model which answers in a single forward pass (~33 ms).
- Laya MLX — Native MLX runtime for Laya, 13.4 ms median end-to-end for a short English decision on Apple Silicon.
- Intern Decision 4B multimodal, Qwen 3.5 fine-tune — Takes a shared state, a schema of questions and optional images and returns an answer distribution for every question in one forward pass.
- DiffusionGemma as Jev — Serve DiffusionGemma-Jev on Cloud Run with an NVIDIA RTX PRO 6000 Blackwell GPU. Promoted by the Google Gemma X/Twitter account, which shared Matt Mastracci’s live evals of Jev vs DiffusionGemma-as-Jev.
- See the GitHub Repo — The example server for Jev-style structured decisions on DiffusionGemma. The vLLM PR has since been merged. See the Red Hat article for running it on vLLM and Red Hat AI.
Collection of Jev’s & scores:
- Leaderboard (multiple open Jev’s) — The Jev Decision Index by Apolinário (multimodalart), hosted on Hugging Face Spaces. It scores 70 open reproductions of Jev on the same benchmark suite, 120K decisions per model across 43 benchmarks.
Research level:
- CLM (Contrastive Language Model) — Two small projection heads on a frozen Qwen3-8B encoder, trained with a contrastive objective connecting states and actions. The authors report it’s on par with Jev on computer-use, gaming and tool-calling tasks with up to 9x lower latency. See the blog post from researchers at Stanford and NVIDIA Research.
- Bespoke Labs Nimble 9B — Apache 2.0 Qwen3.5-9B LoRA adapter which scores the allowed answer tokens directly. Bespoke Labs share how they curated the data, trained and served the model on their GitHub.
- Kev series — Decision models with a pointer head over the answer options, run prefill-only without generating text. Comes in 0.8B, 4B and 9B LoRA adapters on Qwen3.5 as well as Kev-27B, a full fine-tune of Qwen3.8-27B for documents of up to 64k tokens. Based on the architecture post described here: Jev’s architecture unmasked.
- Example use case: Firecrawl re-ranking search results with Jev — TypeSafe’s reranking cookbook took top-1 accuracy on a legal retrieval set from 5% to 18% by scoring a BM25 shortlist with Jev.
Papers
- A better harness for image generation = better outcomes with smaller models — AutoRef optimizes the program around a frozen image generator instead of training it. The discovered harness takes FLUX.2 klein 4B from 5.72 to 7.37 on held-out MultiBanana tasks, above Nano Banana Pro (7.20).
- VisionHOPE — A vision backbone that keeps learning as it reads an image, using five coupled memories which store content and control its own learning rate and retention. “The first generic visual backbone formulated as a self-modifying learning system, in which what the model remembers and how it learns co-evolve within an image.”
- TailSFT: Filtered SFT for improvements in post-training — Filters out already fit sequences during SFT so learning focuses on the tail of the data. On OLMo-3 7B this gives up to 17% absolute pass@16 gains and better starting checkpoints for RL. Related in object detection, FocalLoss, down-weights the loss on well-classified examples so training focuses on the hard ones.
Videos
- Latent Space: Why I couldn’t build Jev at OpenAI - Diogo Almeida, TypeSafe Co-founder & CEO — Diogo talks through System One models, RLCD and why TypeSafe won’t optimize around public benchmarks.
- Jev: ChatGPT Co-Creator’s Answer to RLHF by Diogo Almeida at AI Council — Another talk from the Jev founder, given 4 months before Jev launched. One of my favourite points was about training AI models: “you get what you optimize for”. For example, ChatGPT was optimized for chatting with humans by RLHF (Reinforcement Learning from Human Feedback) so it’s not suited to some tasks which aren’t specifically chat based (enter Jev-style models).
- DHH Rails World 2026 opening keynote — DHH opens Rails World 2026 in Austin with “a keynote on the age of AI agents: why 37signals has gone ‘pencils down’ on handwritten code”.
Misc
Advice of the month: Act with Agency by Mitchell Hashimoto
See you next month!
What a massive month for the AI/ML world in September!
As always, let me know if there's anything you think should be included in a future post.
In the meantime, keep learning, keep creating, keep dancing.
See you next month,
Daniel
By the way, I'm also an instructor with Zero To Mastery Academy teaching people Machine Learning & AI in the most efficient way possible. You can see a few of our courses below or check out all Zero To Mastery courses.
