Independent AI intelligence Two editions daily · ET
FervorAI
Menu

AI Trending Briefing · August 3, 2026 · afternoon edition

Three projects on today's board run frontier-scale models on machines that cannot hold them by streaming weights off NVMe, which moves the binding constraint on local inference from RAM to storage bandwidth and makes every headline memory number misleading on its own.

WASTEKimi K3AirLLMProject Perceptionreverse-skillvideo-uselocal-aiagent-securityfrontier-modelsai-skillsagent-infrastructure

Trending AI Briefing: Monday, August 3, 2026 (afternoon ET)

Three of the sixteen fastest-moving repositories on this afternoon's board make the same claim in different units: a model far larger than your RAM will run on your machine. A 2.78-trillion-parameter Kimi K3 on a 64 GB laptop. The same model in 8.24 GB. A 70B on a 4 GB GPU. All three work by the same trick, keeping only the active slice of weights resident and reading the rest off storage per token. The memory ceiling stopped being the wall. Storage bandwidth became it, and the throughput numbers underneath the headlines are where the honesty lives.

What's hottest in AI news right now

SQLite Cloud released WASTE on July 30, an Apache-2.0 inference engine in C with no runtime dependencies, and it is the most carefully measured of the three. It runs the complete 2.78T Kimi K3, not a distilled or pruned variant, on a 64 GB MacBook Pro with an M5 Pro at roughly 0.45 to 0.62 tokens per second. K3 is a mixture-of-experts model where about 4% of parameters activate per token, so WASTE keeps a 27.28 GB trunk resident, leaves the 896 routed experts on NVMe, and uses leftover RAM as a bounded expert cache. The published 1.42 TB checkpoint converts to a 982 GB container. Each token reads about 17 GB of experts. The internal SSD sustains 12.78 GB/s; a tested USB enclosure managed 0.94 GB/s, which is why the docs tell you flatly to put the container on internal storage. (GitHub, Marco Bambini's release writeup)

The best thing in the repository is not the engine, it is docs/LEARNED.md, a file of things the team believed and then disproved. A three-bit trunk freed six gigabytes and raised the cache hit rate, then collapsed generation quality, because K3 got quantization-aware training on its experts and not on the rest. A Metal backend passed correctness and ran 22% slower than CPU. An index-layout change worth 1.44x in a microbenchmark produced nothing end to end. Their earliest projection said 1 to 1.5 tok/s and the measurement came in near 0.3; the wrong projection is still in the repo with an explanation of why it was wrong.

FareedKhan-dev published kimi-k3-in-c, which pushes the same idea to a harder floor: the same 2.78T model in 8.24 GB of RAM, portable C99, no BLAS, no framework, no GPU. Byte-identical output across every memory configuration is the claimed invariant, and the quickstart is honest about what that costs. The engine moves roughly 135 GB per token at small budgets, and throughput at the 8 GB setting is 33 seconds per token. Setup means a 1.56 TB download across 96 shards plus a trunk-packing step, which is what turns a fixed 115 GB requirement into an adjustable budget. There is no chat template, so output completes your prompt rather than answering it. (GitHub, quickstart)

AirLLM reached the Hacker News front page this afternoon and supplies the ceiling number for the whole category. It layer-streams from disk to fit 70B on a 4 GB GPU with no quantization, distillation, or pruning, and its release notes put Kimi K3 on an RTX 6000 Ada with 48 GB at 292 seconds per token. The comment thread is the useful part. The top reply reads, in full: a lot of these "run 1TB models with 1GB RAM" projects are showing up lately, most seem vibe coded, and probably will not be maintained. That is the correct skepticism, and it applies unevenly across the three. (GitHub, Hacker News)

Microsoft's Project Perception entered public preview today, August 3, eight days after its July 27 announcement alongside MAI-Cyber-1-Flash, the company's first cybersecurity-specialized model. Security work splits across three agent roles that share intelligence: Red simulates attacks and profiles likely threat actors, Blue detects and triages, Green writes and deploys patches. MAI-Cyber-1-Flash descends from the in-house MAI-Thinking-1 lineage and is sized to carry roughly 90% of routine security tasks, escalating only the hardest to GPT-5.4. A major vendor now sells defender agents as a product category rather than a research demo. (Microsoft AI, Microsoft Security, TechCrunch)

zhaoxuya520/reverse-skill sits at number one on the momentum board, and it is the offense to Perception's defense. An MIT-licensed skill router pack for reverse engineering, authorized penetration testing, and security research, it carries more than twenty sub-skills spanning APK teardown, IDA analysis, JavaScript unpacking, firmware, malware, APIs, supply chain, EDR bypass, and exploitation. An agent reads routing.md, classifies the target, and enters the matching playbook instead of improvising commands. It bootstraps missing tools, MCP servers, and scripts on demand, keeps a knowledge base that updates from its own runs, and drives Claude Code, Kiro, Cursor, and Cline. (GitHub, Trendshift)

New tools and features worth actually trying

WASTE, but not with K3. Start with Kimi-Linear 48B: a 19 GB container, a 1.28 GB memory floor, about 8.9 to 10.7 tok/s on the same MacBook, with an OpenAI-compatible server, tools, structured output, and images. That is a usable local model in an afternoon. Honest tradeoff: the K3 path costs 1.42 TB of staging storage, roughly 4.7 hours of conversion, and about 1 TB of internal NVMe you probably want for something else, and at 0.6 tok/s it is a measurement rather than a workflow.

reverse-skill is worth reading whatever your beat, because the routing file is the clearest published example of skill selection by target classification rather than by keyword. Honest tradeoff: this is offensive tooling with EDR-bypass and exploitation playbooks wired into an agent that bootstraps its own toolchain, and installing it means accepting that blast radius on a machine your coding agent already has credentials for.

DeepSeek-Reasonix is the cost-side counterpart to today's memory-side story: a single static Go binary that keeps the system-prompt prefix byte-stable across turns so DeepSeek's automatic prefix cache stays warm, holding 90%+ cache hits on long sessions and cutting input-token cost to roughly a fifth. Honest tradeoff: the design is welded to one provider's caching behavior, and any mid-session mutation of the prefix, a memory write included, throws the savings away. (GitHub)

Trending AI repos on GitHub today

  • sqliteai/waste: C inference engine that streams MoE experts from NVMe to run models far past available RAM. The negative-results doc is worth more than the benchmark.
  • FareedKhan-dev/kimi-k3-in-c: 2.78T Kimi K3 in 8.24 GB of RAM, portable C99, no GPU. The extreme end of the same idea, at 33 s/token.
  • lyogavin/airllm: layer-by-layer streaming to fit 70B on a 4 GB GPU without quantization. Back on the HN front page today and drawing the sharpest skepticism.
  • zhaoxuya520/reverse-skill: security skill router that classifies a target and routes the agent to a playbook. Top of the momentum board, MIT, and unusually broad in scope.
  • esengine/DeepSeek-Reasonix: DeepSeek-native terminal coding agent built around prefix-cache stability. Treats token cost as an architectural constraint.
  • browser-use/video-use: lets Claude Code or Codex drive FFmpeg, subtitle burners, and color grading from a folder of raw footage. The agent reads a 12 KB timestamped transcript instead of frames.
  • stablyai/orca: agent development environment for running a fleet of coding agents in parallel, each in its own git worktree, on desktop, mobile, or VPS.
  • TencentCloud/TencentDB-Agent-Memory: team-level memory hub building Chat Memory, Skill, LLM-Wiki, and Code-Graph assets locally with no external API calls.

Rankings come from Trendshift's daily momentum board. Star totals there and on GitHub disagreed during this scan, so treat placement as direction rather than an audited count.

What actually matters from today's signal

Track storage as an inference parameter this week. That is the real content of all three streaming projects, and it changes what a deployment spec has to say. Four things worth measuring: bytes read per token, 17 GB for K3 under WASTE and roughly 135 GB under kimi-k3-in-c at small budgets; sustained random-read bandwidth on the actual device, because a USB enclosure at 0.94 GB/s against internal NVMe at 12.78 GB/s is a thirteen-fold gap no spec sheet warned about; resident trunk size, which sets the hard floor rather than total model size; and cache sizing, where WASTE measured throughput dropping eightfold as it handed the cache more memory, since the hit rate kept climbing while every hit turned into a page fault.

The counter-signal is that Hacker News reply, and it deserves to be taken seriously rather than dismissed as cynicism. A category is forming where the headline is a memory number and the product is a throughput nobody quotes. The distance between 0.6 tok/s, 33 seconds per token, and 292 seconds per token is the distance between a research result, a demo, and a screenshot. WASTE is the one that publishes what it got wrong, ties every measurement to a commit and a machine, and reports unstable numbers as ranges. Use that as the filter. A local-inference project that will not show you its negative results is quoting a ceiling, not a capability.

One more thing to sit with. Today the number one repository on the board is an agent that reverse engineers binaries and bypasses endpoint detection, and the number one vendor launch is an agent that patches the holes. Both shipped the same week, both are agents, and only one of them requires a purchase order.

Source access notes: Direct GitHub HTML fetches on zhaoxuya520/reverse-skill and kimi-k3-in-c root exceeded the response limit, so those were verified via the Trendshift repository page and the repo's own docs pages instead. Fresh URLs are blocked by a provenance rule this run, so several repos were reached through WebSearch results first. api.github.com was not used per standing policy. The Hacker News front page and Product Hunt were not directly reachable; HN was covered through individual item pages. arXiv new-submission listings returned nothing on the day's beat beyond existing agent-memory survey work. Star counts on Trendshift and GitHub disagreed and are reported as direction only.