The New York Times reports that a Malaysian data center plans to provide computing power to a Chinese technology company. Its operator is a client of Cantor Fitzgerald, the firm Howard Lutnick formerly led and handed to his sons.
The Times says Lutnick has advocated a less stringent version of proposed controls on chip sales to international data centers. The existing arrangement described in the article is legal.
Caveat: Only the article’s preview was accessible, so the complete investigation and any responses beyond that preview could not be checked.
Models split text into tokens, which can be whole words or smaller pieces. Position information helps the model distinguish where those pieces appear.
Raschka contrasts GPT-2’s learned position vectors with rotary position embeddings, a method that incorporates position into attention. He also explains models that omit an explicit position encoding.
The walkthrough includes diagrams, a small PyTorch implementation and a survey of position choices in recent open-weight models, which are downloadable models with licence terms attached.
Caveat: The tutorial assumes familiarity with basic language-model architecture and attention; it is a technical guide, not a new model release.
(HN 175 points · 16 comments · Sebastian Raschka)
Mapping conversations inside Qwen — STALE: the November 13 HN submission describes an exploratory visualization; no new result since earlier editions was established.