Qwen’s new hosted model
- Alibaba releases Qwen4-Max for coding and professional work — Developers get a new paid model to test, with lower published prices and a separate plugin for reviewing work while it runs.
- Max is available through QwenCloud and Qwen’s applications, with text, image and video inputs documented on the model page. Its weights are unavailable; smaller downloadable Qwen4 models are promised later.
- Alibaba describes a 2.4-trillion-parameter backbone, with 71 billion parameters active per token, plus 328 billion embedding parameters. Its advertised context holds one million tokens, the pieces into which models divide text.
- QwenCloud lists prices of $1.50 per million input tokens and $4.50 per million output tokens. Reusing an already processed prompt costs $0.075 per million input tokens; creating an explicit saved prompt costs extra.
- Alibaba reports that average cost on its Work360 professional-task test fell from $4.43 with Qwen3.8-Max to $1.67 with Max, a 62% reduction.
- Artificial Analysis gives it an Intelligence Index score of 43 and reports a weighted cost of $3.27 per task. That is a separate test from Work360.
- The Live Delivery plugin initially supports Codex CLI on Linux and macOS. It saves versions and accepts feedback while an agent, a model running a multi-step task with tools, continues working.
- Caveat: Alibaba’s task-cost savings come from its own Work360 test and do not predict savings on your workload. (HN 594 points · 249 comments · Techmeme · QwenCloud · Artificial Analysis)
Testing browser work and catching regressions
-
WebArena-Pro releases repeatable browser tasks across twenty applications paper notes → — Developers can test browser assistants against resettable applications, including tasks that require listening to audio or inspecting video.
- The released project is a benchmark, a fixed set of tasks for comparing models. Its repository contains 100 development tasks and 300 test tasks across 20 self-hosted applications.
- Each task starts from reset application state. The test set includes 96 cross-application tasks and 79 audio/video tasks; those categories can overlap.
- At a 50-action limit, the paper reports 62.3% success for both Codex with GPT 6.1 Sol and Claude Code with Opus 5.5. Its table and project leaderboard put OpenCode with Kimi 3.1 Pro at 60.7%.
- Python 3.12+, Docker and uv are required for the documented setup. BrowserGym, Harbor and MCP integrations are available; MCP is the standard for connecting agents to tools.
- Caveat: Scores compare complete model-and-tool setups, and some grading depends on model judgments rather than deterministic application checks. (arXiv 2611.17955 · project release dated November 16)
-
A test suite can improve while an assistant loses a tool paper notes → — Tracking results by requirement and behavior can expose failures hidden by an overall pass rate.
- The November 13 paper inserts seven controlled defects into an airline-support assistant and repeats two defect types in retail support. The changes include removing a tool and forcing escalation to a human.
- Removing the cancellation tool broke all three previously passing airline tasks that required it. Yet the total rose from 25 to 27 successes across 50 tasks because unrelated outcomes changed.
- Under the study’s rule requiring a drop greater than ten percentage points and a statistical check, the generated airline suite detected one of seven defects. The hand-written suite detected none.
- Scoring just the affected tasks recovered the tool-removal failure. The paper recommends reporting coverage and checking behavior, such as escalation, alongside requirement-level results.
- Caveat: This is a small, mostly single-run pilot using the same model as assistant, simulated user and grader, with several analyses chosen after the results. (arXiv 2611.17632)
Keeping evidence during long tasks
- A new CarryOn paper preserves exact details during long tasks paper notes → — Keeping a short progress record alongside original evidence offers an alternative to repeatedly summarizing away useful identifiers and records.
- Lihuan Li and colleagues keep original observation fragments linked to their sources. A trained selector chooses what the acting model sees and can restore evidence omitted earlier.
- The selector uses a 0.6-billion-parameter model, trained separately for each test environment and reused across acting models. Selection adds no extra text-generation calls, but still requires local computation.
- Across AppWorld, OfficeBench and Tau2 Telecom with three acting models, the authors report 75.7% mean task success. That exceeds their strongest overall baseline, SWE-Pruner, by 6.7 percentage points.
- Their resource score includes failed tasks but excludes processing by local selection and pruning models. It measures calls and tokens, rather than total hardware cost.
- This is a different team and method from the October CarryOn paper already covered here. The new paper’s linked anonymous code repository could not load.
- Caveat: Each setting has one reported run, and transfer to unfamiliar applications remains untested. (arXiv 2611.17808)
Reviewing AI-written infrastructure
- Nextest’s maintainer shows how human review caught AI-written mistakes — The installer case study shows why extensive generated tests still need a person to check the intended behavior.
- The November 14 account describes using Claude Code and Codex to implement and test Unix and Windows installers. The maintainer designed the installation process and reviewed the code manually.
- The generated tests covered many options but initially missed the default installation path. They also approved redirects from encrypted HTTPS downloads to unencrypted HTTP.
- The maintainer found these problems while writing explanatory comments. The revised scripts reject those redirects and add missing Windows interruption tests.
- Manual review took about 25 hours over four weekends. The merged change contains about 10,000 lines, mostly tests, including failure injection and checks for interrupted downloads.
- Caveat: This is one maintainer’s documented experience, not a controlled measurement of AI coding productivity. (Lobsters 4 points · 0 comments · maintainer account)
Health AI and data handling
- Apple launches a study app for everyday health questions — The study creates a concrete example of health-data sharing with an external AI service, rather than an on-device assistant.
- TechCrunch reports that Apple and OpenEvidence aim to recruit up to 200,000 US participants for more than twelve months. Participants can ask questions, rate answers and optionally share selected Apple Health data.
- The report says participants must be over 21, with an iPhone 15 or later, iOS 26 or later and Apple Intelligence enabled. The App Store listing confirms the app exists and requires iOS 26.
- Apple says data is anonymized before reaching a third-party cloud provider and will not train AI models. TechCrunch reports that OpenEvidence deletes query-and-response data after 24 hours.
- Apple’s linked general study privacy policy says each study’s consent governs specific retention terms. Withdrawal may leave already collected study data in use.
- Caveat: The 24-hour claim concerns OpenEvidence’s query-and-response data, not deletion of all study records or evidence that the app improves health outcomes. (Techmeme · TechCrunch · Apple App Store)
All gathered items - what was cut and why (8)
- When agents should forget → UNVERIFIABLE: the page pairs a November-format arXiv identifier with an August 30 submission date and no later version. Its freshness could not be reconciled; no universal deletion threshold is repeated.
- Hallucination-detector rankings on small local models → UNVERIFIABLE: the page similarly pairs a November-format identifier with an August 31 submission date. A current collection appearance alone does not establish new research.
- A proposed EU model → UNVERIFIABLE: both yesterday’s editions cut the same headline; this collection adds no checked primary release or evaluation.
- Using Copilot with a local model → UNVERIFIABLE: yesterday’s verification attempts could not read the linked setup article, and this run establishes no new development.
- A proposal to open-source AI-generated code → DEDUP: the existing standalone capture already treats the unchanged proposal and its discussion.
- An argument against relying on copyright → OFFSTACK: a substantive general-software commons essay, but its AI references are incidental rather than a new AI development.
- Kindo’s funding announcement → LOW_UTILITY: the collected funding and revenue claims supply no checked new tool or deployment method for this edition.
- FlashMap’s speculative decoding announcement → STALE: the collected post is dated October 28, with no verified newer release in this run.