Running models locally

  • Unsloth updates its smaller Qwen model downloads — People running Qwen3.8-27B locally have new compressed model files to compare against their current setup.

    • The November 11 update applies Unsloth’s Dynamic V3.1 preview to most downloads between two and six bits. Compression stores model weights at lower precision so they fit in less memory.
    • Unsloth left its three-bit versions unchanged because it found little improvement from the new method. It links the downloadable GGUF files and local-running instructions.
    • The company reports improvements of 5–10% on its tests at comparable sizes. Its guide estimates 16–19 GB of combined system and graphics memory for four-bit versions, depending on the setup.
    • Caveat: The quality comparisons are Unsloth’s own tests; memory needs and results will vary with the workload and context length.
    • Background: Dynamic 3.0’s earlier release covered a previous compression method, not this update. (HN 159 points · 48 comments · Unsloth documentation)
  • A library behind local model tools gets a major update — Developers building or connecting local model software should check compatibility before upgrading.

    • GGML 1.0 removes deprecated public interfaces and older masking operations. It changes the remote procedure call protocol, used to connect processes across machines, to reject incompatible peers.
    • Its scheduler skips uploading inputs that the computation will not read. The release also lists performance work for NVIDIA CUDA, Apple Metal, Vulkan and other hardware backends.
    • On Vulkan, an added Intel kernel speeds part of the initial prompt-processing path; the release notes also list fixes for out-of-bounds accesses on Metal and Vulkan.
    • Caveat: These are library changes; the notes do not promise a particular speedup for an existing application or machine. (HN 292 points · 46 comments · GGML release notes)

AI in air traffic

  • Air traffic controllers get flight advice nationwide — Controllers now have a wider deployment of software that suggests changes to flight routes and scheduling.
    • The FAA said on November 10 that SMART would cover all 22 US air-route traffic control centers. The cloud-based system combines weather, flight paths, traffic and staffing data to suggest traffic-management decisions.
    • FAA specialists validate recommendations before forwarding them to facilities, and local controllers decide whether to accept them. The agency says the system cannot take control of an aircraft.
    • The FAA says its initial deployment helped reroute 180 flights, saving an average of 10 minutes and almost 80 gallons of fuel per flight. Those are agency figures, not an independent assessment.
    • Caveat: The controllers’ union told the cited reporting it was not involved in designing, testing or implementing SMART and could not speak to workforce effects. (Techmeme · Ars Technica · FAA statements)
  • Antithesis uses search and a game plan to beat NetHack — The experiment shows how a testing system can explore many possible paths when people specify actions and progress measures.

    • The company says its system found a winning path within five wall-clock minutes for nearly 80% of tested game seeds. Among the seeds that finished, median time was under two minutes.
    • Its approach combines game-specific instructions and progress rewards with search. The account also describes using exploration to find a NetHack bug and AI-assisted testing to investigate related bugs.
    • Caveat: These are the company’s own results using substantial game-specific setup and its testing infrastructure, not a measure of an unaided model playing in one continuous run. (Lobsters 32 points · 3 comments · Antithesis)
  • A clustering method reports faster results on fixed datasets — Developers grouping large collections of vectors can inspect an alternative to repeatedly scanning the whole dataset.

    • K-means groups data points around representative centers. E-means updates those centers using small batches and chooses batch and step sizes from fitted rules.
    • The authors report reaching the comparison method’s 25-pass test error in 1.2–3 passes across their tested scales. They report 10–45 times less elapsed time on GPU against Flash-KMeans, and 8–15 times less on CPU against Faiss.
    • Their code and paper are linked from the project page. Its VIBE tests include text, code, image and multimodal vector datasets.
    • Caveat: These are authors’ comparisons on fixed datasets and chosen implementations; the results do not establish the same gains for a changing dataset or every workload. (Lobsters 5 points · 0 comments · project page)
Skipped this run (4 items)